Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 37299:e9aadee698cf

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
wireproto: add frame flag to denote payloads as CBOR We may eventually want a separate frame type for this. But for now this is the easiest to implement. Differential Revision: https://phab.mercurial-scm.org/D2986
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 28 Mar 2018 13:30:24 -0700
parents 5ef2da00e935
children 36d17f37db91
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 import struct
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
16 from .i18n import _
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
17 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
18 attr,
37290
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
19 cbor,
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
20 )
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 from . import (
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
22 error,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 util,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24 )
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37066
diff changeset
25 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37066
diff changeset
26 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37066
diff changeset
27 )
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
29 FRAME_HEADER_SIZE = 8
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
32 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
33 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
34 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
35
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
36 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
37 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
38 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
39 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
40 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
41
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
42 FRAME_TYPE_COMMAND_REQUEST = 0x01
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 FRAME_TYPE_COMMAND_DATA = 0x03
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
44 FRAME_TYPE_BYTES_RESPONSE = 0x04
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
45 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
46 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291
b0041036214e wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37290
diff changeset
47 FRAME_TYPE_PROGRESS = 0x07
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
48 FRAME_TYPE_STREAM_SETTINGS = 0x08
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 FRAME_TYPES = {
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
51 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52 b'command-data': FRAME_TYPE_COMMAND_DATA,
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
53 b'bytes-response': FRAME_TYPE_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
54 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
55 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291
b0041036214e wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37290
diff changeset
56 b'progress': FRAME_TYPE_PROGRESS,
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
57 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
58 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
60 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
61 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
62 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
63 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
65 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
66 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
67 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
68 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
69 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
80 FLAG_BYTES_RESPONSE_CONTINUATION = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
81 FLAG_BYTES_RESPONSE_EOS = 0x02
37299
e9aadee698cf wireproto: add frame flag to denote payloads as CBOR
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37298
diff changeset
82 FLAG_BYTES_RESPONSE_CBOR = 0x04
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
83
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
84 FLAGS_BYTES_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
85 b'continuation': FLAG_BYTES_RESPONSE_CONTINUATION,
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
86 b'eos': FLAG_BYTES_RESPONSE_EOS,
37299
e9aadee698cf wireproto: add frame flag to denote payloads as CBOR
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37298
diff changeset
87 b'cbor': FLAG_BYTES_RESPONSE_CBOR,
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
88 }
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
89
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
90 FLAG_ERROR_RESPONSE_PROTOCOL = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
91 FLAG_ERROR_RESPONSE_APPLICATION = 0x02
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
92
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
93 FLAGS_ERROR_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
94 b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL,
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
95 b'application': FLAG_ERROR_RESPONSE_APPLICATION,
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
96 }
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
97
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99 FRAME_TYPE_FLAGS = {
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
100 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
101 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
102 FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
103 FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE,
37063
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
104 FRAME_TYPE_TEXT_OUTPUT: {},
37291
b0041036214e wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37290
diff changeset
105 FRAME_TYPE_PROGRESS: {},
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
106 FRAME_TYPE_STREAM_SETTINGS: {},
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
107 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
108
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
109 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
110
37298
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
111 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
112 """Convert a numeric flags value to a human value, using a mapping table."""
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
113 flags = []
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
114 for val, name in sorted({v: k for k, v in mapping.iteritems()}.iteritems()):
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
115 if value & val:
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
116 flags.append(name)
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
117
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
118 return b'|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
119
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
120 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
121 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
122 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
123
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
124 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
125 requestid = attr.ib()
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
126 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
127 streamflags = attr.ib()
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
128 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
129 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
130
37298
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
131 @attr.s(slots=True, repr=False)
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
132 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
133 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
134
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
135 requestid = attr.ib()
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
136 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
137 streamflags = attr.ib()
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
138 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
139 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
140 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
141
37298
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
142 def __repr__(self):
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
143 typename = '<unknown>'
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
144 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
145 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
146 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
147 break
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
148
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
149 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
150 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
151 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
152 humanflags(STREAM_FLAGS, self.streamflags), typename,
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
153 humanflags(FRAME_TYPE_FLAGS[self.typeid], self.flags)))
5ef2da00e935 wireproto: implement custom __repr__ for frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37292
diff changeset
154
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
155 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
156 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
157 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
158 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
159
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
160 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
161 # 16 bits request id
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
162 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
163 # 8 bits stream flags
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
164 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
165 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
166
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168 frame[0:3] = l[0:3]
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
169 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
170 frame[7] = (typeid << 4) | flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
171 frame[8:] = payload
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
172
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173 return frame
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175 def makeframefromhumanstring(s):
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
176 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
177
37290
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
178 DANGER: NOT SAFE TO USE WITH UNTRUSTED INPUT BECAUSE OF POTENTIAL
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
179 eval() USAGE. DO NOT USE IN CORE.
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
180
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
181 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
182
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
183 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
184
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
185 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
186 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
187
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
188 Request ID and stream IDs are integers.
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
189
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
190 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
191 named constant.
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
192
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
193 Flags can be delimited by `|` to bitwise OR them together.
37290
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
194
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
195 If the payload begins with ``cbor:``, the following string will be
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
196 evaluated as Python code and the resulting object will be fed into
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
197 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
198 byte string literal.
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
199 """
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
200 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
201 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
202
2ec1fb9de638 wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
203 requestid = int(requestid)
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
204 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
205
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
206 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
207 for flag in streamflags.split(b'|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
208 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
209 finalstreamflags |= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
210 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
211 finalstreamflags |= int(flag)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
212
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
213 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
214 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
215 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
216 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
217
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
218 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
219 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
220 for flag in frameflags.split(b'|'):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
221 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
222 finalflags |= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
223 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
224 finalflags |= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
225
37290
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
226 if payload.startswith(b'cbor:'):
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
227 payload = cbor.dumps(stringutil.evalpython(payload[5:]), canonical=True)
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
228
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
229 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37289
diff changeset
230 payload = stringutil.unescapestr(payload)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
231
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
232 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
233 streamflags=finalstreamflags, typeid=frametype,
37065
fe4c944f95bb wireproto: use named arguments when passing around frame data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37064
diff changeset
234 flags=finalflags, payload=payload)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
235
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
236 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
237 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
238
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
239 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
240 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
241 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
242 # 24 bits payload length (little endian)
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
243 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
244 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
245 # 8 bits stream flags
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
246 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
247 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
248 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
249 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
250 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
251 typeflags = data[7]
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
252
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
253 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
254 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
255
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
256 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
257 frametype, frameflags)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
258
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
259 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
260 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
261
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
262 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
263 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
264 seen.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
265 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
266 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
267
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
268 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
269
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
270 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
271 return None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
272
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
273 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
274 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
275 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
276
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
277 h = parseheader(header)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
278
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
279 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
280 if len(payload) != h.length:
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
281 raise error.Abort(_('frame length error: expected %d; got %d') %
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
282 (h.length, len(payload)))
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
283
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
284 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
285 payload)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
286
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
287 def createcommandframes(stream, requestid, cmd, args, datafh=None,
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
288 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
289 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
290
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
291 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
292 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
293 """
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
294 data = {b'name': cmd}
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
295 if args:
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
296 data[b'args'] = args
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
297
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
298 data = cbor.dumps(data, canonical=True)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
299
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
300 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
301
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
302 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
303 flags = 0
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
304
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
305 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
306 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
307 flags |= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
308 else:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
309 flags |= FLAG_COMMAND_REQUEST_CONTINUATION
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
310
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
311 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
312 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
313 flags |= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
314
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
315 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
316 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
317
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
318 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
319 flags |= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
320
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
321 yield stream.makeframe(requestid=requestid,
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
322 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
323 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
324 payload=payload)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
325
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
326 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
327 break
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
328
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
329 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
330 while True:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
331 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
332
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
333 done = False
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
334 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
335 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
336 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
337 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
338 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
339 done = True
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
340
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
341 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
342 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
343 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
344 payload=data)
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
345
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
346 if done:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
347 break
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
348
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
349 def createbytesresponseframesfrombytes(stream, requestid, data,
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
350 maxframesize=DEFAULT_MAX_FRAME_SIZE):
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
351 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
352
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
353 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
354 """
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
355
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
356 # Simple case of a single frame.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
357 if len(data) <= maxframesize:
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
358 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
359 typeid=FRAME_TYPE_BYTES_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
360 flags=FLAG_BYTES_RESPONSE_EOS,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
361 payload=data)
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
362 return
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
363
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
364 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
365 while True:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
366 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
367 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
368 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
369
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
370 if done:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
371 flags = FLAG_BYTES_RESPONSE_EOS
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
372 else:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
373 flags = FLAG_BYTES_RESPONSE_CONTINUATION
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
374
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
375 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
376 typeid=FRAME_TYPE_BYTES_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
377 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
378 payload=chunk)
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
379
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
380 if done:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
381 break
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
382
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
383 def createerrorframe(stream, requestid, msg, protocol=False, application=False):
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
384 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
385 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
386
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
387 flags = 0
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
388 if protocol:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
389 flags |= FLAG_ERROR_RESPONSE_PROTOCOL
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
390 if application:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
391 flags |= FLAG_ERROR_RESPONSE_APPLICATION
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
392
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
393 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
394 typeid=FRAME_TYPE_ERROR_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
395 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
396 payload=msg)
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
397
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
398 def createtextoutputframe(stream, requestid, atoms):
37063
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
399 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
400
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
401 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
402
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
403 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
404 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
405 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
406 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
407 """
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
408 bytesleft = DEFAULT_MAX_FRAME_SIZE
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
409 atomchunks = []
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
410
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
411 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
412 if len(args) > 255:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
413 raise ValueError('cannot use more than 255 formatting arguments')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
414 if len(labels) > 255:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
415 raise ValueError('cannot use more than 255 labels')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
416
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
417 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
418
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
419 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
420 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
421 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
422 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
423 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
424 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
425 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
426 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
427
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
428 # Formatting string must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
429 formatting = formatting.decode(r'utf-8', r'replace').encode(r'utf-8')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
430
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
431 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
432 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
433
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
434 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
435 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
436 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
437
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
438 if len(formatting) > 65535:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
439 raise ValueError('formatting string cannot be longer than 64k')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
440
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
441 if any(len(a) > 65535 for a in args):
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
442 raise ValueError('argument string cannot be longer than 64k')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
443
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
444 if any(len(l) > 255 for l in labels):
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
445 raise ValueError('label string cannot be longer than 255 bytes')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
446
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
447 chunks = [
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
448 struct.pack(r'<H', len(formatting)),
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
449 struct.pack(r'<BB', len(labels), len(args)),
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
450 struct.pack(r'<' + r'B' * len(labels), *map(len, labels)),
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
451 struct.pack(r'<' + r'H' * len(args), *map(len, args)),
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
452 ]
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
453 chunks.append(formatting)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
454 chunks.extend(labels)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
455 chunks.extend(args)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
456
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
457 atom = b''.join(chunks)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
458 atomchunks.append(atom)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
459 bytesleft -= len(atom)
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
460
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
461 if bytesleft < 0:
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
462 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
463
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
464 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
465 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
466 flags=0,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
467 payload=b''.join(atomchunks))
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
468
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
469 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
470 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
471
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
472 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
473 self.streamid = streamid
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
474 self._active = False
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
475
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
476 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
477 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
478
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
479 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
480 """
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
481 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
482 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
483 streamflags |= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
484 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
485
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
486 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
487 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
488
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
489 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
490 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
491 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
492 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
493 stream.streamid)
37063
0a6c5cc09a88 wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37061
diff changeset
494
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
495 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
496 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
497
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
498 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
499 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
500 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
501 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
502
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
503 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
504 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
505 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
506 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
507 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
508 *run a command*), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
509
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
510 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
511 this class.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
512
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
513 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
514 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
515 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
516 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
517 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
518 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
519 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
520
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
521 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
522
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
523 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
524 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
525 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
526 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
527
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
528 error
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
529 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
530
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
531 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
532 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
533 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
534
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
535 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
536 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
537 more frames from the client before anything interesting can be done.
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
538
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
539 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
540 Indicates no additional action is required.
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
541
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
542 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
543 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
544
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
545 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
546 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
547 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
548
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
549 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
550 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
551 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
552 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
553
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
554 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
555 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
556 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
557 between who responds to what.
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
558 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
559
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
560 def __init__(self, deferoutput=False):
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
561 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
562
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
563 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
564 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
565 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
566 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
567 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
568 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
569 """
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
570 self._deferoutput = deferoutput
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
571 self._state = 'idle'
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
572 self._nextoutgoingstreamid = 2
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
573 self._bufferedframegens = []
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
574 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
575 self._incomingstreams = {}
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
576 self._outgoingstreams = {}
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
577 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
578 self._receivingcommands = {}
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
579 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
580 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
581 # set.
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
582 self._activecommands = set()
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
583
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
584 def onframerecv(self, frame):
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
585 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
586
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
587 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
588 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
589 """
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
590 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
591 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
592 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
593 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
594 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
595
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
596 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
597 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
598 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
599 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
600 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
601 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
602
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
603 self._incomingstreams[frame.streamid] = stream(frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
604
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
605 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
606 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
607 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
608 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
609 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
610
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
611 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
612 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
613
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
614 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
615 'idle': self._onframeidle,
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
616 'command-receiving': self._onframecommandreceiving,
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
617 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
618 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
619
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
620 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
621 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
622 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
623
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
624 return meth(frame)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
625
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
626 def onbytesresponseready(self, stream, requestid, data):
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
627 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
628
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
629 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
630 """
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
631 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
632
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
633 def sendframes():
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
634 for frame in createbytesresponseframesfrombytes(stream, requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
635 data):
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
636 yield frame
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
637
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
638 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
639
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
640 result = sendframes()
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
641
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
642 if self._deferoutput:
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
643 self._bufferedframegens.append(result)
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
644 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
645 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
646 return 'sendframes', {
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
647 'framegen': result,
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
648 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
649
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
650 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
651 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
652
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
653 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
654 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
655 """
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
656 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
657
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
658 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
659 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
660
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
661 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
662 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
663 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
664 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
665 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
666
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
667 return 'sendframes', {
37059
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37058
diff changeset
668 'framegen': makegen(),
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
669 }
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
670
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
671 def onapplicationerror(self, stream, requestid, msg):
37288
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
672 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37287
diff changeset
673
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
674 return 'sendframes', {
37287
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
675 'framegen': createerrorframe(stream, requestid, msg,
3ed344546d9e wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37087
diff changeset
676 application=True),
37058
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
677 }
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
678
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
679 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
680 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
681 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
682 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
683
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
684 s = stream(streamid)
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
685 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
686
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
687 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
688
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
689 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
690 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
691 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
692 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
693
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
694 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
695 entry = self._receivingcommands[requestid]
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
696
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
697 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
698 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
699 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
700 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
701
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
702 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
703
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
704 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
705 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
706 else:
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
707 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
708
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
709 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
710 entry['payload'].seek(0)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
711 request = cbor.load(entry['payload'])
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
712
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
713 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
714 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
715 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
716 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
717
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
718 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
719 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
720
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
721 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
722 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
723
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
724 return 'runcommand', {
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
725 'requestid': requestid,
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
726 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
727 'args': request[b'args'],
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
728 'data': entry['data'].getvalue() if entry['data'] else None,
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
729 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
730
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
731 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
732 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
733 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
734 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
735
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
736 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
737 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
738 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
739
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
740 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
741 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
742 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
743 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
744 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
745
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
746 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
747 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
748 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
749 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
750 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
751
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
752 def _onframeidle(self, frame):
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
753 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
754 # command request.
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
755 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
756 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
757 return self._makeerrorresult(
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
758 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
759
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
760 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
761 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
762 return res
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
763
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
764 if frame.requestid in self._receivingcommands:
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
765 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
766 return self._makeerrorresult(
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
767 _('request with ID %d already received') % frame.requestid)
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
768
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
769 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
770 self._state = 'errored'
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
771 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
772 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
773
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
774 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
775 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
776 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
777
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
778 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
779 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
780 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
781 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
782
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
783 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
784 payload.write(frame.payload)
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
785
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
786 self._receivingcommands[frame.requestid] = {
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
787 'payload': payload,
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
788 'data': None,
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
789 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
790 'expectingdata': bool(expectingdata),
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
791 }
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
792
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
793 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
794 if not moreframes and not expectingdata:
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
795 return self._makeruncommandresult(frame.requestid)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
796
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
797 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
798 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
799 return self._makewantframeresult()
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
800
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
801 def _onframecommandreceiving(self, frame):
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
802 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
803 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
804 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
805 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
806
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
807 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
808 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
809 return res
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
810
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
811 # All other frames should be related to a command that is currently
37066
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
812 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
813 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
814 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
815 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
816 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
817 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37065
diff changeset
818
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
819 if frame.requestid not in self._receivingcommands:
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
820 self._state = 'errored'
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
821 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
822 _('received frame for request that is not receiving: %d') %
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
823 frame.requestid)
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
824
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
825 entry = self._receivingcommands[frame.requestid]
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
826
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
827 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
828 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
829 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
830
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
831 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
832 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
833 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
834 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
835 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
836
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
837 if expectingdata != entry['expectingdata']:
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
838 self._state = 'errored'
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
839 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
840 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
841
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
842 entry['payload'].write(frame.payload)
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
843
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
844 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
845 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
846
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
847 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
848 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
849
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
850 return self._makewantframeresult()
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
851
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
852 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
853 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
854 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
855 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
856 'received command data frame for request that is not '
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
857 'expecting data: %d') % frame.requestid)
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
858
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
859 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
860 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
861
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
862 return self._handlecommanddataframe(frame, entry)
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
863 else:
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
864 self._state = 'errored'
37292
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
865 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37291
diff changeset
866 'received unexpected frame type: %d') % frame.typeid)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
867
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
868 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
869 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
870
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
871 # TODO support streaming data instead of buffering it.
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
872 entry['data'].write(frame.payload)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
873
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
874 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
875 return self._makewantframeresult()
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
876 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061
c5e9c3b47366 wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37060
diff changeset
877 entry['data'].seek(0)
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
878 return self._makeruncommandresult(frame.requestid)
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
879 else:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
880 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
881 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
882 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
883
37064
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37063
diff changeset
884 def _onframeerrored(self, frame):
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
885 return self._makeerrorresult(_('server already errored'))