mercurial-scm/hg-stable: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 40133:762ef19a07e3

wireprotov2: send protocol settings frame from client Now that we have client and server reactor support for protocol settings and encoding frames, we can start to send them out over the wire! This commit teaches the client reactor to send out a protocol settings frame when needed. The httpv2 peer has been taught to gather a list of supported content encoders and to advertise them through the client reactor. Because the client is now sending new frame types by default, this constitutes a compatibility break in the framing protocol. The media type version has been bumped accordingly. This will ensure existing clients won't attempt to send the new frames to old servers not supporting this explicit media type. I'm not bothering with the BC annotation because everything wireprotov2 is highly experimental and nobody should be running a server yet. Differential Revision: https://phab.mercurial-scm.org/D4922

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Mon, 08 Oct 2018 17:00:16 -0700
parents	e67522413ca8
children	966b5f7fd30b

rev	line source
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	14 import collections
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	17 from .i18n import _
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	18 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	19 attr,
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	20 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 from . import (
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	22 encoding,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	23 error,
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	24 pycompat,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 util,
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	26 wireprototypes,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	28 from .utils import (
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	29 cborutil,
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	30 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	31 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	33 FRAME_HEADER_SIZE = 8
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	36 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	37 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	38 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	39
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	40 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	41 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	42 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	43 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	44 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	45
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	46 FRAME_TYPE_COMMAND_REQUEST = 0x01
37723 e8fba6d578f0 wireprotov2: change frame type value for command data Gregory Szorc <gregory.szorc@gmail.com> parents: 37722 diff changeset	47 FRAME_TYPE_COMMAND_DATA = 0x02
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	48 FRAME_TYPE_COMMAND_RESPONSE = 0x03
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	49 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	50 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	51 FRAME_TYPE_PROGRESS = 0x07
40126 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	52 FRAME_TYPE_SENDER_PROTOCOL_SETTINGS = 0x08
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	53 FRAME_TYPE_STREAM_SETTINGS = 0x09
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	56 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 b'command-data': FRAME_TYPE_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	58 b'command-response': FRAME_TYPE_COMMAND_RESPONSE,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	59 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	60 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	61 b'progress': FRAME_TYPE_PROGRESS,
40126 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	62 b'sender-protocol-settings': FRAME_TYPE_SENDER_PROTOCOL_SETTINGS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	63 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	66 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	68 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	71 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	72 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	73 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	74 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	75 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	86 FLAG_COMMAND_RESPONSE_CONTINUATION = 0x01
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	87 FLAG_COMMAND_RESPONSE_EOS = 0x02
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	88
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	89 FLAGS_COMMAND_RESPONSE = {
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	90 b'continuation': FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	91 b'eos': FLAG_COMMAND_RESPONSE_EOS,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	92 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	93
40126 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	94 FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION = 0x01
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	95 FLAG_SENDER_PROTOCOL_SETTINGS_EOS = 0x02
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	96
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	97 FLAGS_SENDER_PROTOCOL_SETTINGS = {
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	98 b'continuation': FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	99 b'eos': FLAG_SENDER_PROTOCOL_SETTINGS_EOS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	100 }
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	101
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	102 FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION = 0x01
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	103 FLAG_STREAM_ENCODING_SETTINGS_EOS = 0x02
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	104
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	105 FLAGS_STREAM_ENCODING_SETTINGS = {
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	106 b'continuation': FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	107 b'eos': FLAG_STREAM_ENCODING_SETTINGS_EOS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	108 }
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	109
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	111 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	112 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	113 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	114 FRAME_TYPE_COMMAND_RESPONSE: FLAGS_COMMAND_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	115 FRAME_TYPE_ERROR_RESPONSE: {},
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	116 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	117 FRAME_TYPE_PROGRESS: {},
40126 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	118 FRAME_TYPE_SENDER_PROTOCOL_SETTINGS: FLAGS_SENDER_PROTOCOL_SETTINGS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40026 diff changeset	119 FRAME_TYPE_STREAM_SETTINGS: FLAGS_STREAM_ENCODING_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	120 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	121
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	122 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	124 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	125 """Convert a numeric flags value to a human value, using a mapping table."""
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	126 namemap = {v: k for k, v in mapping.iteritems()}
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	127 flags = []
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	128 val = 1
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	129 while value >= val:
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	130 if value & val:
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	131 flags.append(namemap.get(val, '<unknown 0x%02x>' % val))
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	132 val <<= 1
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	133
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	134 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	135
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	136 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	137 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	138 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	139
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	140 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	141 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	142 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	143 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	144 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	145 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	146
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	147 @attr.s(slots=True, repr=False)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	148 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	149 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	150
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	151 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	152 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	153 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	154 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	155 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	156 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	157
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	158 @encoding.strmethod
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	159 def __repr__(self):
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	160 typename = '<unknown 0x%02x>' % self.typeid
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	161 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	162 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	163 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	164 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	165
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	166 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	167 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	168 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	169 humanflags(STREAM_FLAGS, self.streamflags), typename,
37472 2f81926c7f89 wireproto: fix repr(frame) to not crash by unknown type id Yuya Nishihara <yuya@tcha.org> parents: 37319 diff changeset	170 humanflags(FRAME_TYPE_FLAGS.get(self.typeid, {}), self.flags)))
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	171
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	172 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	177 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	178 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	179 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	180 # 8 bits stream flags
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	181 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	182 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	183
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	186 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	187 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	188 frame[8:] = payload
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 def makeframefromhumanstring(s):
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	193 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	194
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	195 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	196
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	197 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	198
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	202 Request ID and stream IDs are integers.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	203
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	204 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	205 named constant.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	206
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	208
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	209 If the payload begins with ``cbor:``, the following string will be
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	210 evaluated as Python literal and the resulting object will be fed into
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	211 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	212 byte string literal.
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	214 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	215 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	216
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	217 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	218 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	219
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	220 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	221 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	222 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	223 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	224 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	225 finalstreamflags \|= int(flag)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	240 if payload.startswith(b'cbor:'):
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	241 payload = b''.join(cborutil.streamencode(
36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	242 stringutil.evalpythonliteral(payload[5:])))
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	243
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	244 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	245 payload = stringutil.unescapestr(payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	247 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	248 streamflags=finalstreamflags, typeid=frametype,
37065 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37064 diff changeset	249 flags=finalflags, payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	251 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	252 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	253
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	254 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	255 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	256 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	257 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	258 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	259 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	260 # 8 bits stream flags
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	261 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	262 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	263 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	264 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	265 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	266 typeflags = data[7]
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	267
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	268 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	269 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	270
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	271 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	272 frametype, frameflags)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	273
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	274 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	275 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	276
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	277 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	278 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	279 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	280 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	281 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	282
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	283 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	284
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	285 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	286 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	287
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	288 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	289 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	290 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	291
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	292 h = parseheader(header)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	293
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	294 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	295 if len(payload) != h.length:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	296 raise error.Abort(_('frame length error: expected %d; got %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	297 (h.length, len(payload)))
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	298
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	299 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	300 payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	301
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	302 def createcommandframes(stream, requestid, cmd, args, datafh=None,
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	303 maxframesize=DEFAULT_MAX_FRAME_SIZE,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	304 redirect=None):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	310 data = {b'name': cmd}
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	312 data[b'args'] = args
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	314 if redirect:
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	315 data[b'redirect'] = redirect
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	316
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	317 data = b''.join(cborutil.streamencode(data))
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	319 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	321 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	322 flags = 0
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	324 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	325 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	326 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	327 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	328 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	330 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	331 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	332 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	334 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	335 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	336
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	337 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	338 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	339
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	340 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	341 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	342 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	343 payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	345 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	346 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	347
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	359
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	360 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	361 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	362 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	363 payload=data)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366 break
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	367
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	368 def createcommandresponseframesfrombytes(stream, requestid, data,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	369 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	370 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	371
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	372 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	373 """
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	374 # Automatically send the overall CBOR response map.
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	375 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	376 if len(overall) > maxframesize:
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	377 raise error.ProgrammingError('not yet implemented')
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	378
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	379 # Simple case where we can fit the full response in a single frame.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	380 if len(overall) + len(data) <= maxframesize:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	381 flags = FLAG_COMMAND_RESPONSE_EOS
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	382 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	383 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	384 flags=flags,
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	385 payload=overall + data)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	386 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	387
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	388 # It's easier to send the overall CBOR map in its own frame than to track
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	389 # offsets.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	390 yield stream.makeframe(requestid=requestid,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	391 typeid=FRAME_TYPE_COMMAND_RESPONSE,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	392 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	393 payload=overall)
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	394
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	395 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	396 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	397 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	398 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	399 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	400
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	401 if done:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	402 flags = FLAG_COMMAND_RESPONSE_EOS
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	403 else:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	404 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	405
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	406 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	407 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	408 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	409 payload=chunk)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	410
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	411 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	412 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	413
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	414 def createbytesresponseframesfromgen(stream, requestid, gen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	415 maxframesize=DEFAULT_MAX_FRAME_SIZE):
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	416 """Generator of frames from a generator of byte chunks.
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	417
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	418 This assumes that another frame will follow whatever this emits. i.e.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	419 this always emits the continuation flag and never emits the end-of-stream
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	420 flag.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	421 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	422 cb = util.chunkbuffer(gen)
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	423 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	424
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	425 while True:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	426 chunk = cb.read(maxframesize)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	427 if not chunk:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	428 break
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	429
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	430 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	431 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	432 flags=flags,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	433 payload=chunk)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	434
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	435 flags \|= FLAG_COMMAND_RESPONSE_CONTINUATION
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	436
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	437 def createcommandresponseokframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	438 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	439
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	440 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	441 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	442 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	443 payload=overall)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	444
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	445 def createcommandresponseeosframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	446 """Create an empty payload frame representing command end-of-stream."""
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	447 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	448 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	449 flags=FLAG_COMMAND_RESPONSE_EOS,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	450 payload=b'')
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	451
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	452 def createalternatelocationresponseframe(stream, requestid, location):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	453 data = {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	454 b'status': b'redirect',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	455 b'location': {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	456 b'url': location.url,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	457 b'mediatype': location.mediatype,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	458 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	459 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	460
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	461 for a in (r'size', r'fullhashes', r'fullhashseed', r'serverdercerts',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	462 r'servercadercerts'):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	463 value = getattr(location, a)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	464 if value is not None:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	465 data[b'location'][pycompat.bytestr(a)] = value
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	466
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	467 return stream.makeframe(requestid=requestid,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	468 typeid=FRAME_TYPE_COMMAND_RESPONSE,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	469 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	470 payload=b''.join(cborutil.streamencode(data)))
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	471
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	472 def createcommanderrorresponse(stream, requestid, message, args=None):
39503 43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39465 diff changeset	473 # TODO should this be using a list of {'msg': ..., 'args': {}} so atom
43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39465 diff changeset	474 # formatting works consistently?
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	475 m = {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	476 b'status': b'error',
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	477 b'error': {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	478 b'message': message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	479 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	480 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	481
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	482 if args:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	483 m[b'error'][b'args'] = args
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	484
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	485 overall = b''.join(cborutil.streamencode(m))
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	486
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	487 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	488 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	489 flags=FLAG_COMMAND_RESPONSE_EOS,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	490 payload=overall)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	491
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	492 def createerrorframe(stream, requestid, msg, errtype):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	493 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	494 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	495
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	496 payload = b''.join(cborutil.streamencode({
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	497 b'type': errtype,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	498 b'message': [{b'msg': msg}],
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	499 }))
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	500
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	501 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	502 typeid=FRAME_TYPE_ERROR_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	503 flags=0,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	504 payload=payload)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	505
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	506 def createtextoutputframe(stream, requestid, atoms,
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	507 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	508 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	509
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	510 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	511
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	512 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	513 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	514 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	515 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	516 """
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	517 atomdicts = []
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	518
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	519 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	520 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	521
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	522 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	523 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	524 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	525 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	526 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	527 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	528 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	529 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	530
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	531 # Formatting string must be ASCII.
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	532 formatting = formatting.decode(r'ascii', r'replace').encode(r'ascii')
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	533
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	534 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	535 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	536
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	537 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	538 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	539 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	540
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	541 atom = {b'msg': formatting}
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	542 if args:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	543 atom[b'args'] = args
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	544 if labels:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	545 atom[b'labels'] = labels
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	546
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	547 atomdicts.append(atom)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	548
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	549 payload = b''.join(cborutil.streamencode(atomdicts))
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	550
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	551 if len(payload) > maxframesize:
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	552 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	553
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	554 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	555 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	556 flags=0,
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	557 payload=payload)
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	558
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	559 class bufferingcommandresponseemitter(object):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	560 """Helper object to emit command response frames intelligently.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	561
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	562 Raw command response data is likely emitted in chunks much smaller
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	563 than what can fit in a single frame. This class exists to buffer
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	564 chunks until enough data is available to fit in a single frame.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	565
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	566 TODO we'll need something like this when compression is supported.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	567 So it might make sense to implement this functionality at the stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	568 level.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	569 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	570 def __init__(self, stream, requestid, maxframesize=DEFAULT_MAX_FRAME_SIZE):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	571 self._stream = stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	572 self._requestid = requestid
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	573 self._maxsize = maxframesize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	574 self._chunks = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	575 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	576
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	577 def send(self, data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	578 """Send new data for emission.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	579
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	580 Is a generator of new frames that were derived from the new input.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	581
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	582 If the special input ``None`` is received, flushes all buffered
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	583 data to frames.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	584 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	585
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	586 if data is None:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	587 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	588 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	589 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	590
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	591 # There is a ton of potential to do more complicated things here.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	592 # Our immediate goal is to coalesce small chunks into big frames,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	593 # not achieve the fewest number of frames possible. So we go with
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	594 # a simple implementation:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	595 #
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	596 # * If a chunk is too large for a frame, we flush and emit frames
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	597 # for the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	598 # * If a chunk can be buffered without total buffered size limits
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	599 # being exceeded, we do that.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	600 # * If a chunk causes us to go over our buffering limit, we flush
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	601 # and then buffer the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	602
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	603 if len(data) > self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	604 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	605 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	606
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	607 # Now emit frames for the big chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	608 offset = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	609 while True:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	610 chunk = data[offset:offset + self._maxsize]
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	611 offset += len(chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	612
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	613 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	614 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	615 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	616 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	617 payload=chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	618
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	619 if offset == len(data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	620 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	621
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	622 # If we don't have enough to constitute a full frame, buffer and
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	623 # return.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	624 if len(data) + self._chunkssize < self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	625 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	626 self._chunkssize += len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	627 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	628
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	629 # Else flush what we have and buffer the new chunk. We could do
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	630 # something more intelligent here, like break the chunk. Let's
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	631 # keep things simple for now.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	632 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	633 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	634
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	635 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	636 self._chunkssize = len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	637
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	638 def _flush(self):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	639 payload = b''.join(self._chunks)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	640 assert len(payload) <= self._maxsize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	641
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	642 self._chunks[:] = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	643 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	644
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	645 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	646 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	647 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	648 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	649 payload=payload)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	650
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	651 # TODO consider defining encoders/decoders using the util.compressionengine
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	652 # mechanism.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	653
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	654 class identityencoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	655 """Encoder for the "identity" stream encoding profile."""
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	656 def __init__(self, ui):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	657 pass
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	658
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	659 def encode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	660 return data
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	661
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	662 def flush(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	663 return b''
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	664
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	665 def finish(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	666 return b''
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	667
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	668 class identitydecoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	669 """Decoder for the "identity" stream encoding profile."""
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	670
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	671 def __init__(self, ui, extraobjs):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	672 if extraobjs:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	673 raise error.Abort(_('identity decoder received unexpected '
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	674 'additional values'))
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	675
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	676 def decode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	677 return data
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	678
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	679 class zlibencoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	680 def __init__(self, ui):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	681 import zlib
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	682 self._zlib = zlib
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	683 self._compressor = zlib.compressobj()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	684
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	685 def encode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	686 return self._compressor.compress(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	687
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	688 def flush(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	689 # Z_SYNC_FLUSH doesn't reset compression context, which is
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	690 # what we want.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	691 return self._compressor.flush(self._zlib.Z_SYNC_FLUSH)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	692
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	693 def finish(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	694 res = self._compressor.flush(self._zlib.Z_FINISH)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	695 self._compressor = None
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	696 return res
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	697
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	698 class zlibdecoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	699 def __init__(self, ui, extraobjs):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	700 import zlib
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	701
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	702 if extraobjs:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	703 raise error.Abort(_('zlib decoder received unexpected '
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	704 'additional values'))
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	705
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	706 self._decompressor = zlib.decompressobj()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	707
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	708 def decode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	709 # Python 2's zlib module doesn't use the buffer protocol and can't
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	710 # handle all bytes-like types.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	711 if not pycompat.ispy3 and isinstance(data, bytearray):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	712 data = bytes(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	713
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	714 return self._decompressor.decompress(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	715
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	716 class zstdbaseencoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	717 def __init__(self, level):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	718 from . import zstd
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	719
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	720 self._zstd = zstd
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	721 cctx = zstd.ZstdCompressor(level=level)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	722 self._compressor = cctx.compressobj()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	723
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	724 def encode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	725 return self._compressor.compress(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	726
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	727 def flush(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	728 # COMPRESSOBJ_FLUSH_BLOCK flushes all data previously fed into the
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	729 # compressor and allows a decompressor to access all encoded data
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	730 # up to this point.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	731 return self._compressor.flush(self._zstd.COMPRESSOBJ_FLUSH_BLOCK)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	732
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	733 def finish(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	734 res = self._compressor.flush(self._zstd.COMPRESSOBJ_FLUSH_FINISH)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	735 self._compressor = None
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	736 return res
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	737
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	738 class zstd8mbencoder(zstdbaseencoder):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	739 def __init__(self, ui):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	740 super(zstd8mbencoder, self).__init__(3)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	741
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	742 class zstdbasedecoder(object):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	743 def __init__(self, maxwindowsize):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	744 from . import zstd
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	745 dctx = zstd.ZstdDecompressor(max_window_size=maxwindowsize)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	746 self._decompressor = dctx.decompressobj()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	747
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	748 def decode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	749 return self._decompressor.decompress(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	750
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	751 class zstd8mbdecoder(zstdbasedecoder):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	752 def __init__(self, ui, extraobjs):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	753 if extraobjs:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	754 raise error.Abort(_('zstd8mb decoder received unexpected '
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	755 'additional values'))
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	756
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	757 super(zstd8mbdecoder, self).__init__(maxwindowsize=8 * 1048576)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	758
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	759 # We lazily populate this to avoid excessive module imports when importing
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	760 # this module.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	761 STREAM_ENCODERS = {}
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	762 STREAM_ENCODERS_ORDER = []
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	763
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	764 def populatestreamencoders():
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	765 if STREAM_ENCODERS:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	766 return
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	767
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	768 try:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	769 from . import zstd
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	770 zstd.__version__
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	771 except ImportError:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	772 zstd = None
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	773
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	774 # zstandard is fastest and is preferred.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	775 if zstd:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	776 STREAM_ENCODERS[b'zstd-8mb'] = (zstd8mbencoder, zstd8mbdecoder)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	777 STREAM_ENCODERS_ORDER.append(b'zstd-8mb')
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	778
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	779 STREAM_ENCODERS[b'zlib'] = (zlibencoder, zlibdecoder)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	780 STREAM_ENCODERS_ORDER.append(b'zlib')
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	781
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	782 STREAM_ENCODERS[b'identity'] = (identityencoder, identitydecoder)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	783 STREAM_ENCODERS_ORDER.append(b'identity')
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	784
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	785 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	786 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	787
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	788 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	789 self.streamid = streamid
37655 b9502b5f2066 wireprotoframing: use value passed into function Gregory Szorc <gregory.szorc@gmail.com> parents: 37544 diff changeset	790 self._active = active
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	791
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	792 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	793 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	794
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	795 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	796 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	797 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	798 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	799 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	800 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	801
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	802 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	803 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	804
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	805 class inputstream(stream):
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	806 """Represents a stream used for receiving data."""
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	807
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	808 def __init__(self, streamid, active=False):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	809 super(inputstream, self).__init__(streamid, active=active)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	810 self._decoder = None
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	811
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	812 def setdecoder(self, ui, name, extraobjs):
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	813 """Set the decoder for this stream.
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	814
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	815 Receives the stream profile name and any additional CBOR objects
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	816 decoded from the stream encoding settings frame payloads.
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	817 """
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	818 if name not in STREAM_ENCODERS:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	819 raise error.Abort(_('unknown stream decoder: %s') % name)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	820
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	821 self._decoder = STREAM_ENCODERS[name][1](ui, extraobjs)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	822
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	823 def decode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	824 # Default is identity decoder. We don't bother instantiating one
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	825 # because it is trivial.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	826 if not self._decoder:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	827 return data
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	828
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	829 return self._decoder.decode(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	830
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	831 def flush(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	832 if not self._decoder:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	833 return b''
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	834
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	835 return self._decoder.flush()
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	836
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	837 class outputstream(stream):
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	838 """Represents a stream used for sending data."""
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	839
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	840 def __init__(self, streamid, active=False):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	841 super(outputstream, self).__init__(streamid, active=active)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	842 self._encoder = None
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	843
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	844 def setencoder(self, ui, name):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	845 """Set the encoder for this stream.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	846
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	847 Receives the stream profile name.
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	848 """
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	849 if name not in STREAM_ENCODERS:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	850 raise error.Abort(_('unknown stream encoder: %s') % name)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	851
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	852 self._encoder = STREAM_ENCODERS[name][0](ui)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	853
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	854 def encode(self, data):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	855 if not self._encoder:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	856 return data
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	857
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	858 return self._encoder.encode(data)
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	859
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	860 def flush(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	861 if not self._encoder:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	862 return b''
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	863
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	864 return self._encoder.flush()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	865
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	866 def finish(self):
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	867 if not self._encoder:
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	868 return b''
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	869
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	870 self._encoder.finish()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	871
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	872 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	873 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	874 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	875 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	876 stream.streamid)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	877
40127 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	878 DEFAULT_PROTOCOL_SETTINGS = {
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	879 'contentencodings': [b'identity'],
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	880 }
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	881
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	882 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	883 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	884
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	885 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	886 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	887 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	888 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	889
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	890 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	891 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	892 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	893 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	894 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	895 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	896
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	897 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	898 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	899
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	900 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	901 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	902 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	903 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	904 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	905 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	906 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	907
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	908 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	909
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	910 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	911 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	912 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	913 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	914
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	915 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	916 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	917
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	918 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	919 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	920 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	921
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	922 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	923 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	924 more frames from the client before anything interesting can be done.
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	925
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	926 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	927 Indicates no additional action is required.
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	928
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	929 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	930 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	931
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	932 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	933 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	934 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	935
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	936 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	937 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	938 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	939 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	940
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	941 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	942 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	943 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	944 between who responds to what.
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	945 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	946
40130 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	947 def __init__(self, ui, deferoutput=False):
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	948 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	949
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	950 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	951 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	952 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	953 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	954 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	955 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	956 """
40130 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	957 self._ui = ui
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	958 self._deferoutput = deferoutput
40127 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	959 self._state = 'initial'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	960 self._nextoutgoingstreamid = 2
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	961 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	962 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	963 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	964 self._outgoingstreams = {}
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	965 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	966 self._receivingcommands = {}
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	967 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	968 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	969 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	970 self._activecommands = set()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	971
40127 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	972 self._protocolsettingsdecoder = None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	973
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	974 # Sender protocol settings are optional. Set implied default values.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	975 self._sendersettings = dict(DEFAULT_PROTOCOL_SETTINGS)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	976
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	977 populatestreamencoders()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	978
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	979 def onframerecv(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	980 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	981
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	982 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	983 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	984 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	985 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	986 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	987 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	988 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	989 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	990
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	991 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	992 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	993 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	994 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	995 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	996 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	997
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	998 self._incomingstreams[frame.streamid] = inputstream(frame.streamid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	999
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1000 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1001 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1002 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1003 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1004 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1005
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1006 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1007 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1008
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1009 handlers = {
40127 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1010 'initial': self._onframeinitial,
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1011 'protocol-settings-receiving': self._onframeprotocolsettings,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1012 'idle': self._onframeidle,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1013 'command-receiving': self._onframecommandreceiving,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1014 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1015 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1016
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1017 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1018 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1019 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1020
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1021 return meth(frame)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1022
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1023 def oncommandresponseready(self, stream, requestid, data):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1024 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1025
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1026 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1027 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1028 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1029
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1030 def sendframes():
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1031 for frame in createcommandresponseframesfrombytes(stream, requestid,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1032 data):
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1033 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1034
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1035 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1036
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1037 result = sendframes()
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1038
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1039 if self._deferoutput:
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1040 self._bufferedframegens.append(result)
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1041 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1042 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1043 return 'sendframes', {
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1044 'framegen': result,
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1045 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1046
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1047 def oncommandresponsereadyobjects(self, stream, requestid, objs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1048 """Signal that objects are ready to be sent to the client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1049
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1050 ``objs`` is an iterable of objects (typically a generator) that will
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1051 be encoded via CBOR and added to frames, which will be sent to the
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1052 client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1053 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1054 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1055
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1056 # We need to take care over exception handling. Uncaught exceptions
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1057 # when generating frames could lead to premature end of the frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1058 # stream and the possibility of the server or client process getting
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1059 # in a bad state.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1060 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1061 # Keep in mind that if ``objs`` is a generator, advancing it could
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1062 # raise exceptions that originated in e.g. wire protocol command
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1063 # functions. That is why we differentiate between exceptions raised
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1064 # when iterating versus other exceptions that occur.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1065 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1066 # In all cases, when the function finishes, the request is fully
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1067 # handled and no new frames for it should be seen.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1068
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1069 def sendframes():
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1070 emitted = False
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1071 alternatelocationsent = False
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1072 emitter = bufferingcommandresponseemitter(stream, requestid)
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1073 while True:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1074 try:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1075 o = next(objs)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1076 except StopIteration:
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1077 for frame in emitter.send(None):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1078 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1079
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1080 if emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1081 yield createcommandresponseeosframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1082 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1083
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1084 except error.WireprotoCommandError as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1085 for frame in createcommanderrorresponse(
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1086 stream, requestid, e.message, e.messageargs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1087 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1088 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1089
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1090 except Exception as e:
39843 bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	1091 for frame in createerrorframe(
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	1092 stream, requestid, '%s' % stringutil.forcebytestr(e),
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	1093 errtype='server'):
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	1094
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1095 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1096
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1097 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1098
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1099 try:
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1100 # Alternate location responses can only be the first and
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1101 # only object in the output stream.
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1102 if isinstance(o, wireprototypes.alternatelocationresponse):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1103 if emitted:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1104 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1105 'alternatelocationresponse seen after initial '
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1106 'output object')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1107
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1108 yield createalternatelocationresponseframe(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1109 stream, requestid, o)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1110
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1111 alternatelocationsent = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1112 emitted = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1113 continue
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1114
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1115 if alternatelocationsent:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1116 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1117 'object follows alternatelocationresponse')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1118
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1119 if not emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1120 yield createcommandresponseokframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1121 emitted = True
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1122
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1123 # Objects emitted by command functions can be serializable
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1124 # data structures or special types.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1125 # TODO consider extracting the content normalization to a
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1126 # standalone function, as it may be useful for e.g. cachers.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1127
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1128 # A pre-encoded object is sent directly to the emitter.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1129 if isinstance(o, wireprototypes.encodedresponse):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1130 for frame in emitter.send(o.data):
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1131 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	1132
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1133 # A regular object is CBOR encoded.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1134 else:
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1135 for chunk in cborutil.streamencode(o):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1136 for frame in emitter.send(chunk):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1137 yield frame
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	1138
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1139 except Exception as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1140 for frame in createerrorframe(stream, requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1141 '%s' % e,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1142 errtype='server'):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1143 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1144
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	1145 break
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1146
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1147 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1148
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1149 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1150
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1151 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1152 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1153
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1154 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1155 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1156 """
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1157 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1158
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1159 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1160 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1161
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1162 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1163 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1164 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1165 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1166 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1167
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1168 return 'sendframes', {
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	1169 'framegen': makegen(),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1170 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1171
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1172 def _handlesendframes(self, framegen):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1173 if self._deferoutput:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1174 self._bufferedframegens.append(framegen)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1175 return 'noop', {}
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1176 else:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1177 return 'sendframes', {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1178 'framegen': framegen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1179 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1180
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1181 def onservererror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1182 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	1183
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1184 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1185 for frame in createerrorframe(stream, requestid, msg,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1186 errtype='server'):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1187 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1188
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1189 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1190
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1191 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1192
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1193 def oncommanderror(self, stream, requestid, message, args=None):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1194 """Called when a command encountered an error before sending output."""
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1195 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1196
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1197 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1198 for frame in createcommanderrorresponse(stream, requestid, message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1199 args):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1200 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1201
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1202 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1203
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1204 return self._handlesendframes(sendframes())
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	1205
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1206 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1207 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1208 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1209 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1210
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	1211 s = outputstream(streamid)
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1212 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1213
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1214 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1215
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1216 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1217 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1218 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1219 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1220
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1221 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1222 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1223
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1224 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1225 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1226 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1227 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1228
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1229 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1230
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1231 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1232 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1233 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1234 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1235
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1236 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1237 entry['payload'].seek(0)
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1238 request = cborutil.decodeall(entry['payload'].getvalue())[0]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1239
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1240 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1241 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1242 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1243 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1244
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1245 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1246 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1247
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1248 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1249 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1250
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1251 return 'runcommand', {
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1252 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1253 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1254 'args': request[b'args'],
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1255 'redirect': request.get(b'redirect'),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1256 'data': entry['data'].getvalue() if entry['data'] else None,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1257 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1258
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1259 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1260 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1261 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1262 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1263
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1264 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1265 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1266 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1267
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1268 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1269 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1270 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1271 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1272 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1273
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1274 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1275 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1276 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1277 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1278 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1279
40127 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1280 def _onframeinitial(self, frame):
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1281 # Called when we receive a frame when in the "initial" state.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1282 if frame.typeid == FRAME_TYPE_SENDER_PROTOCOL_SETTINGS:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1283 self._state = 'protocol-settings-receiving'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1284 self._protocolsettingsdecoder = cborutil.bufferingdecoder()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1285 return self._onframeprotocolsettings(frame)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1286
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1287 elif frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1288 self._state = 'idle'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1289 return self._onframeidle(frame)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1290
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1291 else:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1292 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1293 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1294 _('expected sender protocol settings or command request '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1295 'frame; got %d') % frame.typeid)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1296
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1297 def _onframeprotocolsettings(self, frame):
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1298 assert self._state == 'protocol-settings-receiving'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1299 assert self._protocolsettingsdecoder is not None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1300
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1301 if frame.typeid != FRAME_TYPE_SENDER_PROTOCOL_SETTINGS:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1302 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1303 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1304 _('expected sender protocol settings frame; got %d') %
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1305 frame.typeid)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1306
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1307 more = frame.flags & FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1308 eos = frame.flags & FLAG_SENDER_PROTOCOL_SETTINGS_EOS
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1309
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1310 if more and eos:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1311 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1312 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1313 _('sender protocol settings frame cannot have both '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1314 'continuation and end of stream flags set'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1315
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1316 if not more and not eos:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1317 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1318 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1319 _('sender protocol settings frame must have continuation or '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1320 'end of stream flag set'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1321
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1322 # TODO establish limits for maximum amount of data that can be
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1323 # buffered.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1324 try:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1325 self._protocolsettingsdecoder.decode(frame.payload)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1326 except Exception as e:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1327 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1328 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1329 _('error decoding CBOR from sender protocol settings frame: %s')
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1330 % stringutil.forcebytestr(e))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1331
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1332 if more:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1333 return self._makewantframeresult()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1334
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1335 assert eos
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1336
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1337 decoded = self._protocolsettingsdecoder.getavailable()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1338 self._protocolsettingsdecoder = None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1339
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1340 if not decoded:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1341 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1342 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1343 _('sender protocol settings frame did not contain CBOR data'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1344 elif len(decoded) > 1:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1345 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1346 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1347 _('sender protocol settings frame contained multiple CBOR '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1348 'values'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1349
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1350 d = decoded[0]
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1351
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1352 if b'contentencodings' in d:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1353 self._sendersettings['contentencodings'] = d[b'contentencodings']
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1354
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1355 self._state = 'idle'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1356
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1357 return self._makewantframeresult()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1358
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1359 def _onframeidle(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1360 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1361 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1362 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1363 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1364 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1365 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1366
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1367 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1368 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1369 return res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1370
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1371 if frame.requestid in self._receivingcommands:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1372 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1373 return self._makeerrorresult(
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1374 _('request with ID %d already received') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1375
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1376 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1377 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1378 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1379 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1380
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1381 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1382 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1383 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1384
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1385 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1386 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1387 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1388 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1389
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1390 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1391 payload.write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1392
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1393 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1394 'payload': payload,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1395 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1396 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1397 'expectingdata': bool(expectingdata),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1398 }
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1399
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1400 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1401 if not moreframes and not expectingdata:
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1402 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1403
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1404 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1405 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1406 return self._makewantframeresult()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1407
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1408 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1409 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1410 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1411 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1412 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1413
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1414 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1415 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1416 return res
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1417
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1418 # All other frames should be related to a command that is currently
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1419 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1420 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1421 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1422 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1423 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1424 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1425
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1426 if frame.requestid not in self._receivingcommands:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1427 self._state = 'errored'
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1428 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1429 _('received frame for request that is not receiving: %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1430 frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1431
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1432 entry = self._receivingcommands[frame.requestid]
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1433
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1434 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1435 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1436 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1437
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1438 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1439 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1440 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1441 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1442 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1443
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1444 if expectingdata != entry['expectingdata']:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1445 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1446 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1447 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1448
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1449 entry['payload'].write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1450
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1451 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1452 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1453
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1454 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1455 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1456
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1457 return self._makewantframeresult()
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1458
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1459 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1460 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1461 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1462 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1463 'received command data frame for request that is not '
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1464 'expecting data: %d') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1465
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1466 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1467 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1468
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1469 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1470 else:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1471 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1472 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1473 'received unexpected frame type: %d') % frame.typeid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1474
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1475 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1476 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1477
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1478 # TODO support streaming data instead of buffering it.
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1479 entry['data'].write(frame.payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1480
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1481 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1482 return self._makewantframeresult()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1483 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1484 entry['data'].seek(0)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1485 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1486 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1487 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1488 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1489 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1490
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1491 def _onframeerrored(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1492 return self._makeerrorresult(_('server already errored'))
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1493
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1494 class commandrequest(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1495 """Represents a request to run a command."""
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1496
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1497 def __init__(self, requestid, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1498 self.requestid = requestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1499 self.name = name
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1500 self.args = args
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1501 self.datafh = datafh
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1502 self.redirect = redirect
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1503 self.state = 'pending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1504
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1505 class clientreactor(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1506 """Holds state of a client issuing frame-based protocol requests.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1507
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1508 This is like ``serverreactor`` but for client-side state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1509
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1510 Each instance is bound to the lifetime of a connection. For persistent
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1511 connection transports using e.g. TCP sockets and speaking the raw
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1512 framing protocol, there will be a single instance for the lifetime of
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1513 the TCP socket. For transports where there are multiple discrete
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1514 interactions (say tunneled within in HTTP request), there will be a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1515 separate instance for each distinct interaction.
40128 080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1516
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1517 Consumers are expected to tell instances when events occur by calling
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1518 various methods. These methods return a 2-tuple describing any follow-up
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1519 action(s) to take. The first element is the name of an action to
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1520 perform. The second is a data structure (usually a dict) specific to
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1521 that action that contains more information. e.g. if the reactor wants
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1522 to send frames to the server, the data structure will contain a reference
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1523 to those frames.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1524
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1525 Valid actions that consumers can be instructed to take are:
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1526
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1527 noop
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1528 Indicates no additional action is required.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1529
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1530 sendframes
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1531 Indicates that frames should be sent to the server. The ``framegen``
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1532 key contains a generator of frames that should be sent. The reactor
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1533 assumes that all frames in this generator are sent to the server.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1534
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1535 error
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1536 Indicates that an error occurred. The ``message`` key contains an
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1537 error message describing the failure.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1538
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1539 responsedata
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1540 Indicates a response to a previously-issued command was received.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1541
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1542 The ``request`` key contains the ``commandrequest`` instance that
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1543 represents the request this data is for.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1544
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1545 The ``data`` key contains the decoded data from the server.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1546
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1547 ``expectmore`` and ``eos`` evaluate to True when more response data
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1548 is expected to follow or we're at the end of the response stream,
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1549 respectively.
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1550 """
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1551 def __init__(self, ui, hasmultiplesend=False, buffersends=True,
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1552 clientcontentencoders=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1553 """Create a new instance.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1554
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1555 ``hasmultiplesend`` indicates whether multiple sends are supported
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1556 by the transport. When True, it is possible to send commands immediately
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1557 instead of buffering until the caller signals an intent to finish a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1558 send operation.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1559
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1560 ``buffercommands`` indicates whether sends should be buffered until the
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1561 last request has been issued.
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1562
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1563 ``clientcontentencoders`` is an iterable of content encoders the client
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1564 will advertise to the server and that the server can use for encoding
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1565 data. If not defined, the client will not advertise content encoders
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1566 to the server.
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1567 """
40130 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	1568 self._ui = ui
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1569 self._hasmultiplesend = hasmultiplesend
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1570 self._buffersends = buffersends
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1571 self._clientcontentencoders = clientcontentencoders
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1572
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1573 self._canissuecommands = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1574 self._cansend = True
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1575 self._protocolsettingssent = False
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1576
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1577 self._nextrequestid = 1
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1578 # We only support a single outgoing stream for now.
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	1579 self._outgoingstream = outputstream(1)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1580 self._pendingrequests = collections.deque()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1581 self._activerequests = {}
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1582 self._incomingstreams = {}
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1583 self._streamsettingsdecoders = {}
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1584
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1585 populatestreamencoders()
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1586
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1587 def callcommand(self, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1588 """Request that a command be executed.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1589
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1590 Receives the command name, a dict of arguments to pass to the command,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1591 and an optional file object containing the raw data for the command.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1592
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1593 Returns a 3-tuple of (request, action, action data).
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1594 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1595 if not self._canissuecommands:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1596 raise error.ProgrammingError('cannot issue new commands')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1597
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1598 requestid = self._nextrequestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1599 self._nextrequestid += 2
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1600
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1601 request = commandrequest(requestid, name, args, datafh=datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1602 redirect=redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1603
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1604 if self._buffersends:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1605 self._pendingrequests.append(request)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1606 return request, 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1607 else:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1608 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1609 raise error.ProgrammingError('sends cannot be performed on '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1610 'this instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1611
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1612 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1613 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1614 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1615
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1616 return request, 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1617 'framegen': self._makecommandframes(request),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1618 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1619
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1620 def flushcommands(self):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1621 """Request that all queued commands be sent.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1622
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1623 If any commands are buffered, this will instruct the caller to send
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1624 them over the wire. If no commands are buffered it instructs the client
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1625 to no-op.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1626
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1627 If instances aren't configured for multiple sends, no new command
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1628 requests are allowed after this is called.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1629 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1630 if not self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1631 return 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1632
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1633 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1634 raise error.ProgrammingError('sends cannot be performed on this '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1635 'instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1636
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1637 # If the instance only allows sending once, mark that we have fired
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1638 # our one shot.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1639 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1640 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1641 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1642
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1643 def makeframes():
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1644 while self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1645 request = self._pendingrequests.popleft()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1646 for frame in self._makecommandframes(request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1647 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1648
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1649 return 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1650 'framegen': makeframes(),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1651 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1652
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1653 def _makecommandframes(self, request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1654 """Emit frames to issue a command request.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1655
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1656 As a side-effect, update request accounting to reflect its changed
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1657 state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1658 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1659 self._activerequests[request.requestid] = request
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1660 request.state = 'sending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1661
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1662 if not self._protocolsettingssent and self._clientcontentencoders:
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1663 self._protocolsettingssent = True
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1664
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1665 payload = b''.join(cborutil.streamencode({
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1666 b'contentencodings': self._clientcontentencoders,
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1667 }))
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1668
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1669 yield self._outgoingstream.makeframe(
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1670 requestid=request.requestid,
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1671 typeid=FRAME_TYPE_SENDER_PROTOCOL_SETTINGS,
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1672 flags=FLAG_SENDER_PROTOCOL_SETTINGS_EOS,
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1673 payload=payload)
762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40132 diff changeset	1674
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1675 res = createcommandframes(self._outgoingstream,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1676 request.requestid,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1677 request.name,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1678 request.args,
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1679 datafh=request.datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1680 redirect=request.redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1681
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1682 for frame in res:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1683 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1684
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1685 request.state = 'sent'
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1686
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1687 def onframerecv(self, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1688 """Process a frame that has been received off the wire.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1689
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1690 Returns a 2-tuple of (action, meta) describing further action the
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1691 caller needs to take as a result of receiving this frame.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1692 """
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1693 if frame.streamid % 2:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1694 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1695 'message': (
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1696 _('received frame with odd numbered stream ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1697 frame.streamid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1698 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1699
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1700 if frame.streamid not in self._incomingstreams:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1701 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1702 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1703 'message': _('received frame on unknown stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1704 'without beginning of stream flag set'),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1705 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1706
40131 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	1707 self._incomingstreams[frame.streamid] = inputstream(
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40130 diff changeset	1708 frame.streamid)
37656 e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1709
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1710 stream = self._incomingstreams[frame.streamid]
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1711
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1712 # If the payload is encoded, ask the stream to decode it. We
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1713 # merely substitute the decoded result into the frame payload as
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1714 # if it had been transferred all along.
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1715 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1716 frame.payload = stream.decode(frame.payload)
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1717
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1718 if frame.streamflags & STREAM_FLAG_END_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1719 del self._incomingstreams[frame.streamid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1720
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1721 if frame.typeid == FRAME_TYPE_STREAM_SETTINGS:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1722 return self._onstreamsettingsframe(frame)
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1723
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1724 if frame.requestid not in self._activerequests:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1725 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1726 'message': (_('received frame for inactive request ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1727 frame.requestid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1728 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1729
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1730 request = self._activerequests[frame.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1731 request.state = 'receiving'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1732
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1733 handlers = {
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1734 FRAME_TYPE_COMMAND_RESPONSE: self._oncommandresponseframe,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1735 FRAME_TYPE_ERROR_RESPONSE: self._onerrorresponseframe,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1736 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1737
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1738 meth = handlers.get(frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1739 if not meth:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1740 raise error.ProgrammingError('unhandled frame type: %d' %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1741 frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1742
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1743 return meth(request, frame)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1744
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1745 def _onstreamsettingsframe(self, frame):
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1746 assert frame.typeid == FRAME_TYPE_STREAM_SETTINGS
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1747
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1748 more = frame.flags & FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1749 eos = frame.flags & FLAG_STREAM_ENCODING_SETTINGS_EOS
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1750
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1751 if more and eos:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1752 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1753 'message': (_('stream encoding settings frame cannot have both '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1754 'continuation and end of stream flags set')),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1755 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1756
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1757 if not more and not eos:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1758 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1759 'message': _('stream encoding settings frame must have '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1760 'continuation or end of stream flag set'),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1761 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1762
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1763 if frame.streamid not in self._streamsettingsdecoders:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1764 decoder = cborutil.bufferingdecoder()
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1765 self._streamsettingsdecoders[frame.streamid] = decoder
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1766
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1767 decoder = self._streamsettingsdecoders[frame.streamid]
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1768
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1769 try:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1770 decoder.decode(frame.payload)
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1771 except Exception as e:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1772 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1773 'message': (_('error decoding CBOR from stream encoding '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1774 'settings frame: %s') %
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1775 stringutil.forcebytestr(e)),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1776 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1777
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1778 if more:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1779 return 'noop', {}
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1780
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1781 assert eos
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1782
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1783 decoded = decoder.getavailable()
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1784 del self._streamsettingsdecoders[frame.streamid]
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1785
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1786 if not decoded:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1787 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1788 'message': _('stream encoding settings frame did not contain '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1789 'CBOR data'),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1790 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1791
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1792 try:
40132 e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1793 self._incomingstreams[frame.streamid].setdecoder(self._ui,
e67522413ca8 wireprotov2: define and use stream encoders Gregory Szorc <gregory.szorc@gmail.com> parents: 40131 diff changeset	1794 decoded[0],
40129 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1795 decoded[1:])
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1796 except Exception as e:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1797 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1798 'message': (_('error setting stream decoder: %s') %
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1799 stringutil.forcebytestr(e)),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1800 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1801
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1802 return 'noop', {}
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1803
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1804 def _oncommandresponseframe(self, request, frame):
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1805 if frame.flags & FLAG_COMMAND_RESPONSE_EOS:
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1806 request.state = 'received'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1807 del self._activerequests[request.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1808
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1809 return 'responsedata', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1810 'request': request,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1811 'expectmore': frame.flags & FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1812 'eos': frame.flags & FLAG_COMMAND_RESPONSE_EOS,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1813 'data': frame.payload,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1814 }
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1815
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1816 def _onerrorresponseframe(self, request, frame):
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1817 request.state = 'errored'
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1818 del self._activerequests[request.requestid]
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1819
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1820 # The payload should be a CBOR map.
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1821 m = cborutil.decodeall(frame.payload)[0]
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1822
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1823 return 'error', {
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1824 'request': request,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1825 'type': m['type'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1826 'message': m['message'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1827 }

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 40133:762ef19a07e3