Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/wireprotoframing.py @ 37292:3d0e2cd86e05
wireproto: use CBOR for command requests
Now that we're using CBOR in the new wire protocol, let's convert
command requests to it.
Before I wrote this patch and was even thinking about CBOR, I was
thinking about how commands should be issued and came to the
conclusion that we didn't need separate frames to represent the
command name from its arguments. I already had a partially
completed patch prepared to merge the frames.
But with CBOR, it makes the implementation a bit simpler because
we don't need to roll our own serialization.
The changes here are a bit invasive. I tried to split this into
multiple commits to make it easier to review. But it was just too
hard.
* "command name" and "command argument" frames have been collapsed
into a "command request" frame.
* The flags for this new frame are totally different.
* Frame processing has been overhauled to reflect the new order
of things.
* Test fallout was significant. A handful of tests were removed.
Altogether, I think the new code is simpler. We don't have
complicated state around receiving commands. We're either receiving
command request frames or command data frames. We /could/
potentially collapse command data frames into command request
frames. Although I'd have to think a bit more about this before
I do it.
Differential Revision: https://phab.mercurial-scm.org/D2951
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Mon, 26 Mar 2018 14:34:32 -0700 |
parents | b0041036214e |
children | 5ef2da00e935 |
rev | line source |
---|---|
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # wireprotoframing.py - unified framing protocol for wire protocol |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 # |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 # This file contains functionality to support the unified frame-based wire |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 # protocol. For details about the protocol, see |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
10 # `hg help internals.wireprotocol`. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
11 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
12 from __future__ import absolute_import |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
13 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
14 import struct |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
15 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
16 from .i18n import _ |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
17 from .thirdparty import ( |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
18 attr, |
37290
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
19 cbor, |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
20 ) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
21 from . import ( |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
22 error, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
23 util, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
24 ) |
37087
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
37066
diff
changeset
|
25 from .utils import ( |
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
37066
diff
changeset
|
26 stringutil, |
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
37066
diff
changeset
|
27 ) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
28 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
29 FRAME_HEADER_SIZE = 8 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
30 DEFAULT_MAX_FRAME_SIZE = 32768 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
31 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
32 STREAM_FLAG_BEGIN_STREAM = 0x01 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
33 STREAM_FLAG_END_STREAM = 0x02 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
34 STREAM_FLAG_ENCODING_APPLIED = 0x04 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
35 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
36 STREAM_FLAGS = { |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
37 b'stream-begin': STREAM_FLAG_BEGIN_STREAM, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
38 b'stream-end': STREAM_FLAG_END_STREAM, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
39 b'encoded': STREAM_FLAG_ENCODING_APPLIED, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
40 } |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
41 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
42 FRAME_TYPE_COMMAND_REQUEST = 0x01 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
43 FRAME_TYPE_COMMAND_DATA = 0x03 |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
44 FRAME_TYPE_BYTES_RESPONSE = 0x04 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
45 FRAME_TYPE_ERROR_RESPONSE = 0x05 |
37063
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
46 FRAME_TYPE_TEXT_OUTPUT = 0x06 |
37291
b0041036214e
wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37290
diff
changeset
|
47 FRAME_TYPE_PROGRESS = 0x07 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
48 FRAME_TYPE_STREAM_SETTINGS = 0x08 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
49 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
50 FRAME_TYPES = { |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
51 b'command-request': FRAME_TYPE_COMMAND_REQUEST, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
52 b'command-data': FRAME_TYPE_COMMAND_DATA, |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
53 b'bytes-response': FRAME_TYPE_BYTES_RESPONSE, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
54 b'error-response': FRAME_TYPE_ERROR_RESPONSE, |
37063
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
55 b'text-output': FRAME_TYPE_TEXT_OUTPUT, |
37291
b0041036214e
wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37290
diff
changeset
|
56 b'progress': FRAME_TYPE_PROGRESS, |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
57 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
58 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
59 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
60 FLAG_COMMAND_REQUEST_NEW = 0x01 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
61 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
62 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
63 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
64 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
65 FLAGS_COMMAND_REQUEST = { |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
66 b'new': FLAG_COMMAND_REQUEST_NEW, |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
67 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION, |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
68 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES, |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
69 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
70 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
71 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
72 FLAG_COMMAND_DATA_CONTINUATION = 0x01 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
73 FLAG_COMMAND_DATA_EOS = 0x02 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
74 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
75 FLAGS_COMMAND_DATA = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
76 b'continuation': FLAG_COMMAND_DATA_CONTINUATION, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
77 b'eos': FLAG_COMMAND_DATA_EOS, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
78 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
79 |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
80 FLAG_BYTES_RESPONSE_CONTINUATION = 0x01 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
81 FLAG_BYTES_RESPONSE_EOS = 0x02 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
82 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
83 FLAGS_BYTES_RESPONSE = { |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
84 b'continuation': FLAG_BYTES_RESPONSE_CONTINUATION, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
85 b'eos': FLAG_BYTES_RESPONSE_EOS, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
86 } |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
87 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
88 FLAG_ERROR_RESPONSE_PROTOCOL = 0x01 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
89 FLAG_ERROR_RESPONSE_APPLICATION = 0x02 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
90 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
91 FLAGS_ERROR_RESPONSE = { |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
92 b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
93 b'application': FLAG_ERROR_RESPONSE_APPLICATION, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
94 } |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
95 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
96 # Maps frame types to their available flags. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
97 FRAME_TYPE_FLAGS = { |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
98 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
99 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA, |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
100 FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE, |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
101 FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE, |
37063
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
102 FRAME_TYPE_TEXT_OUTPUT: {}, |
37291
b0041036214e
wireproto: define frame to represent progress updates
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37290
diff
changeset
|
103 FRAME_TYPE_PROGRESS: {}, |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
104 FRAME_TYPE_STREAM_SETTINGS: {}, |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
105 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
106 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
107 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH') |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
108 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
109 @attr.s(slots=True) |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
110 class frameheader(object): |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
111 """Represents the data in a frame header.""" |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
112 |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
113 length = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
114 requestid = attr.ib() |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
115 streamid = attr.ib() |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
116 streamflags = attr.ib() |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
117 typeid = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
118 flags = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
119 |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
120 @attr.s(slots=True) |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
121 class frame(object): |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
122 """Represents a parsed frame.""" |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
123 |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
124 requestid = attr.ib() |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
125 streamid = attr.ib() |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
126 streamflags = attr.ib() |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
127 typeid = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
128 flags = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
129 payload = attr.ib() |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
130 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
131 def makeframe(requestid, streamid, streamflags, typeid, flags, payload): |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
132 """Assemble a frame into a byte array.""" |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
133 # TODO assert size of payload. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
134 frame = bytearray(FRAME_HEADER_SIZE + len(payload)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
135 |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
136 # 24 bits length |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
137 # 16 bits request id |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
138 # 8 bits stream id |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
139 # 8 bits stream flags |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
140 # 4 bits type |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
141 # 4 bits flags |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
142 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
143 l = struct.pack(r'<I', len(payload)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
144 frame[0:3] = l[0:3] |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
145 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
146 frame[7] = (typeid << 4) | flags |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
147 frame[8:] = payload |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
148 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
149 return frame |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
150 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
151 def makeframefromhumanstring(s): |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
152 """Create a frame from a human readable string |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
153 |
37290
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
154 DANGER: NOT SAFE TO USE WITH UNTRUSTED INPUT BECAUSE OF POTENTIAL |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
155 eval() USAGE. DO NOT USE IN CORE. |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
156 |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
157 Strings have the form: |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
158 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
159 <request-id> <stream-id> <stream-flags> <type> <flags> <payload> |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
160 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
161 This can be used by user-facing applications and tests for creating |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
162 frames easily without having to type out a bunch of constants. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
163 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
164 Request ID and stream IDs are integers. |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
165 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
166 Stream flags, frame type, and flags can be specified by integer or |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
167 named constant. |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
168 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
169 Flags can be delimited by `|` to bitwise OR them together. |
37290
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
170 |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
171 If the payload begins with ``cbor:``, the following string will be |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
172 evaluated as Python code and the resulting object will be fed into |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
173 a CBOR encoder. Otherwise, the payload is interpreted as a Python |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
174 byte string literal. |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
175 """ |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
176 fields = s.split(b' ', 5) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
177 requestid, streamid, streamflags, frametype, frameflags, payload = fields |
37060
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
178 |
2ec1fb9de638
wireproto: add request IDs to frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37059
diff
changeset
|
179 requestid = int(requestid) |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
180 streamid = int(streamid) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
181 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
182 finalstreamflags = 0 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
183 for flag in streamflags.split(b'|'): |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
184 if flag in STREAM_FLAGS: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
185 finalstreamflags |= STREAM_FLAGS[flag] |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
186 else: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
187 finalstreamflags |= int(flag) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
188 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
189 if frametype in FRAME_TYPES: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
190 frametype = FRAME_TYPES[frametype] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
191 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
192 frametype = int(frametype) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
193 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
194 finalflags = 0 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
195 validflags = FRAME_TYPE_FLAGS[frametype] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
196 for flag in frameflags.split(b'|'): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
197 if flag in validflags: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
198 finalflags |= validflags[flag] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
199 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
200 finalflags |= int(flag) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
201 |
37290
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
202 if payload.startswith(b'cbor:'): |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
203 payload = cbor.dumps(stringutil.evalpython(payload[5:]), canonical=True) |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
204 |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
205 else: |
cc5a040fe150
wireproto: syntax for encoding CBOR into frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37289
diff
changeset
|
206 payload = stringutil.unescapestr(payload) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
207 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
208 return makeframe(requestid=requestid, streamid=streamid, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
209 streamflags=finalstreamflags, typeid=frametype, |
37065
fe4c944f95bb
wireproto: use named arguments when passing around frame data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37064
diff
changeset
|
210 flags=finalflags, payload=payload) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
211 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
212 def parseheader(data): |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
213 """Parse a unified framing protocol frame header from a buffer. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
214 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
215 The header is expected to be in the buffer at offset 0 and the |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
216 buffer is expected to be large enough to hold a full header. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
217 """ |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
218 # 24 bits payload length (little endian) |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
219 # 16 bits request ID |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
220 # 8 bits stream ID |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
221 # 8 bits stream flags |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
222 # 4 bits frame type |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
223 # 4 bits frame flags |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
224 # ... payload |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
225 framelength = data[0] + 256 * data[1] + 16384 * data[2] |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
226 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
227 typeflags = data[7] |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
228 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
229 frametype = (typeflags & 0xf0) >> 4 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
230 frameflags = typeflags & 0x0f |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
231 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
232 return frameheader(framelength, requestid, streamid, streamflags, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
233 frametype, frameflags) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
234 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
235 def readframe(fh): |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
236 """Read a unified framing protocol frame from a file object. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
237 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
238 Returns a 3-tuple of (type, flags, payload) for the decoded frame or |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
239 None if no frame is available. May raise if a malformed frame is |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
240 seen. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
241 """ |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
242 header = bytearray(FRAME_HEADER_SIZE) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
243 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
244 readcount = fh.readinto(header) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
245 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
246 if readcount == 0: |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
247 return None |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
248 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
249 if readcount != FRAME_HEADER_SIZE: |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
250 raise error.Abort(_('received incomplete frame: got %d bytes: %s') % |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
251 (readcount, header)) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
252 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
253 h = parseheader(header) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
254 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
255 payload = fh.read(h.length) |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
256 if len(payload) != h.length: |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
257 raise error.Abort(_('frame length error: expected %d; got %d') % |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
258 (h.length, len(payload))) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
259 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
260 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
261 payload) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
262 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
263 def createcommandframes(stream, requestid, cmd, args, datafh=None, |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
264 maxframesize=DEFAULT_MAX_FRAME_SIZE): |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
265 """Create frames necessary to transmit a request to run a command. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
266 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
267 This is a generator of bytearrays. Each item represents a frame |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
268 ready to be sent over the wire to a peer. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
269 """ |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
270 data = {b'name': cmd} |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
271 if args: |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
272 data[b'args'] = args |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
273 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
274 data = cbor.dumps(data, canonical=True) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
275 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
276 offset = 0 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
277 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
278 while True: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
279 flags = 0 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
280 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
281 # Must set new or continuation flag. |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
282 if not offset: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
283 flags |= FLAG_COMMAND_REQUEST_NEW |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
284 else: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
285 flags |= FLAG_COMMAND_REQUEST_CONTINUATION |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
286 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
287 # Data frames is set on all frames. |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
288 if datafh: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
289 flags |= FLAG_COMMAND_REQUEST_EXPECT_DATA |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
290 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
291 payload = data[offset:offset + maxframesize] |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
292 offset += len(payload) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
293 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
294 if len(payload) == maxframesize and offset < len(data): |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
295 flags |= FLAG_COMMAND_REQUEST_MORE_FRAMES |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
296 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
297 yield stream.makeframe(requestid=requestid, |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
298 typeid=FRAME_TYPE_COMMAND_REQUEST, |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
299 flags=flags, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
300 payload=payload) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
301 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
302 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES): |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
303 break |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
304 |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
305 if datafh: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
306 while True: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
307 data = datafh.read(DEFAULT_MAX_FRAME_SIZE) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
308 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
309 done = False |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
310 if len(data) == DEFAULT_MAX_FRAME_SIZE: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
311 flags = FLAG_COMMAND_DATA_CONTINUATION |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
312 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
313 flags = FLAG_COMMAND_DATA_EOS |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
314 assert datafh.read(1) == b'' |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
315 done = True |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
316 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
317 yield stream.makeframe(requestid=requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
318 typeid=FRAME_TYPE_COMMAND_DATA, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
319 flags=flags, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
320 payload=data) |
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
321 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
322 if done: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
323 break |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
324 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
325 def createbytesresponseframesfrombytes(stream, requestid, data, |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
326 maxframesize=DEFAULT_MAX_FRAME_SIZE): |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
327 """Create a raw frame to send a bytes response from static bytes input. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
328 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
329 Returns a generator of bytearrays. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
330 """ |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
331 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
332 # Simple case of a single frame. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
333 if len(data) <= maxframesize: |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
334 yield stream.makeframe(requestid=requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
335 typeid=FRAME_TYPE_BYTES_RESPONSE, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
336 flags=FLAG_BYTES_RESPONSE_EOS, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
337 payload=data) |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
338 return |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
339 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
340 offset = 0 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
341 while True: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
342 chunk = data[offset:offset + maxframesize] |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
343 offset += len(chunk) |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
344 done = offset == len(data) |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
345 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
346 if done: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
347 flags = FLAG_BYTES_RESPONSE_EOS |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
348 else: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
349 flags = FLAG_BYTES_RESPONSE_CONTINUATION |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
350 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
351 yield stream.makeframe(requestid=requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
352 typeid=FRAME_TYPE_BYTES_RESPONSE, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
353 flags=flags, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
354 payload=chunk) |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
355 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
356 if done: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
357 break |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
358 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
359 def createerrorframe(stream, requestid, msg, protocol=False, application=False): |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
360 # TODO properly handle frame size limits. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
361 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
362 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
363 flags = 0 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
364 if protocol: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
365 flags |= FLAG_ERROR_RESPONSE_PROTOCOL |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
366 if application: |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
367 flags |= FLAG_ERROR_RESPONSE_APPLICATION |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
368 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
369 yield stream.makeframe(requestid=requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
370 typeid=FRAME_TYPE_ERROR_RESPONSE, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
371 flags=flags, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
372 payload=msg) |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
373 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
374 def createtextoutputframe(stream, requestid, atoms): |
37063
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
375 """Create a text output frame to render text to people. |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
376 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
377 ``atoms`` is a 3-tuple of (formatting string, args, labels). |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
378 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
379 The formatting string contains ``%s`` tokens to be replaced by the |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
380 corresponding indexed entry in ``args``. ``labels`` is an iterable of |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
381 formatters to be applied at rendering time. In terms of the ``ui`` |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
382 class, each atom corresponds to a ``ui.write()``. |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
383 """ |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
384 bytesleft = DEFAULT_MAX_FRAME_SIZE |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
385 atomchunks = [] |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
386 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
387 for (formatting, args, labels) in atoms: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
388 if len(args) > 255: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
389 raise ValueError('cannot use more than 255 formatting arguments') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
390 if len(labels) > 255: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
391 raise ValueError('cannot use more than 255 labels') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
392 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
393 # TODO look for localstr, other types here? |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
394 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
395 if not isinstance(formatting, bytes): |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
396 raise ValueError('must use bytes formatting strings') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
397 for arg in args: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
398 if not isinstance(arg, bytes): |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
399 raise ValueError('must use bytes for arguments') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
400 for label in labels: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
401 if not isinstance(label, bytes): |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
402 raise ValueError('must use bytes for labels') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
403 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
404 # Formatting string must be UTF-8. |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
405 formatting = formatting.decode(r'utf-8', r'replace').encode(r'utf-8') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
406 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
407 # Arguments must be UTF-8. |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
408 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args] |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
409 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
410 # Labels must be ASCII. |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
411 labels = [l.decode(r'ascii', r'strict').encode(r'ascii') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
412 for l in labels] |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
413 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
414 if len(formatting) > 65535: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
415 raise ValueError('formatting string cannot be longer than 64k') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
416 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
417 if any(len(a) > 65535 for a in args): |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
418 raise ValueError('argument string cannot be longer than 64k') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
419 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
420 if any(len(l) > 255 for l in labels): |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
421 raise ValueError('label string cannot be longer than 255 bytes') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
422 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
423 chunks = [ |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
424 struct.pack(r'<H', len(formatting)), |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
425 struct.pack(r'<BB', len(labels), len(args)), |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
426 struct.pack(r'<' + r'B' * len(labels), *map(len, labels)), |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
427 struct.pack(r'<' + r'H' * len(args), *map(len, args)), |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
428 ] |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
429 chunks.append(formatting) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
430 chunks.extend(labels) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
431 chunks.extend(args) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
432 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
433 atom = b''.join(chunks) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
434 atomchunks.append(atom) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
435 bytesleft -= len(atom) |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
436 |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
437 if bytesleft < 0: |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
438 raise ValueError('cannot encode data in a single frame') |
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
439 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
440 yield stream.makeframe(requestid=requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
441 typeid=FRAME_TYPE_TEXT_OUTPUT, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
442 flags=0, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
443 payload=b''.join(atomchunks)) |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
444 |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
445 class stream(object): |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
446 """Represents a logical unidirectional series of frames.""" |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
447 |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
448 def __init__(self, streamid, active=False): |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
449 self.streamid = streamid |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
450 self._active = False |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
451 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
452 def makeframe(self, requestid, typeid, flags, payload): |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
453 """Create a frame to be sent out over this stream. |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
454 |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
455 Only returns the frame instance. Does not actually send it. |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
456 """ |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
457 streamflags = 0 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
458 if not self._active: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
459 streamflags |= STREAM_FLAG_BEGIN_STREAM |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
460 self._active = True |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
461 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
462 return makeframe(requestid, self.streamid, streamflags, typeid, flags, |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
463 payload) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
464 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
465 def ensureserverstream(stream): |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
466 if stream.streamid % 2: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
467 raise error.ProgrammingError('server should only write to even ' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
468 'numbered streams; %d is not even' % |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
469 stream.streamid) |
37063
0a6c5cc09a88
wireproto: define human output side channel frame
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37061
diff
changeset
|
470 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
471 class serverreactor(object): |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
472 """Holds state of a server handling frame-based protocol requests. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
473 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
474 This class is the "brain" of the unified frame-based protocol server |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
475 component. While the protocol is stateless from the perspective of |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
476 requests/commands, something needs to track which frames have been |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
477 received, what frames to expect, etc. This class is that thing. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
478 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
479 Instances are modeled as a state machine of sorts. Instances are also |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
480 reactionary to external events. The point of this class is to encapsulate |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
481 the state of the connection and the exchange of frames, not to perform |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
482 work. Instead, callers tell this class when something occurs, like a |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
483 frame arriving. If that activity is worthy of a follow-up action (say |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
484 *run a command*), the return value of that handler will say so. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
485 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
486 I/O and CPU intensive operations are purposefully delegated outside of |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
487 this class. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
488 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
489 Consumers are expected to tell instances when events occur. They do so by |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
490 calling the various ``on*`` methods. These methods return a 2-tuple |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
491 describing any follow-up action(s) to take. The first element is the |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
492 name of an action to perform. The second is a data structure (usually |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
493 a dict) specific to that action that contains more information. e.g. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
494 if the server wants to send frames back to the client, the data structure |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
495 will contain a reference to those frames. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
496 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
497 Valid actions that consumers can be instructed to take are: |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
498 |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
499 sendframes |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
500 Indicates that frames should be sent to the client. The ``framegen`` |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
501 key contains a generator of frames that should be sent. The server |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
502 assumes that all frames are sent to the client. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
503 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
504 error |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
505 Indicates that an error occurred. Consumer should probably abort. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
506 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
507 runcommand |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
508 Indicates that the consumer should run a wire protocol command. Details |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
509 of the command to run are given in the data structure. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
510 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
511 wantframe |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
512 Indicates that nothing of interest happened and the server is waiting on |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
513 more frames from the client before anything interesting can be done. |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
514 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
515 noop |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
516 Indicates no additional action is required. |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
517 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
518 Known Issues |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
519 ------------ |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
520 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
521 There are no limits to the number of partially received commands or their |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
522 size. A malicious client could stream command request data and exhaust the |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
523 server's memory. |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
524 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
525 Partially received commands are not acted upon when end of input is |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
526 reached. Should the server error if it receives a partial request? |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
527 Should the client send a message to abort a partially transmitted request |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
528 to facilitate graceful shutdown? |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
529 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
530 Active requests that haven't been responded to aren't tracked. This means |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
531 that if we receive a command and instruct its dispatch, another command |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
532 with its request ID can come in over the wire and there will be a race |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
533 between who responds to what. |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
534 """ |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
535 |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
536 def __init__(self, deferoutput=False): |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
537 """Construct a new server reactor. |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
538 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
539 ``deferoutput`` can be used to indicate that no output frames should be |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
540 instructed to be sent until input has been exhausted. In this mode, |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
541 events that would normally generate output frames (such as a command |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
542 response being ready) will instead defer instructing the consumer to |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
543 send those frames. This is useful for half-duplex transports where the |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
544 sender cannot receive until all data has been transmitted. |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
545 """ |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
546 self._deferoutput = deferoutput |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
547 self._state = 'idle' |
37289
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
548 self._nextoutgoingstreamid = 2 |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
549 self._bufferedframegens = [] |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
550 # stream id -> stream instance for all active streams from the client. |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
551 self._incomingstreams = {} |
37289
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
552 self._outgoingstreams = {} |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
553 # request id -> dict of commands that are actively being received. |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
554 self._receivingcommands = {} |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
555 # Request IDs that have been received and are actively being processed. |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
556 # Once all output for a request has been sent, it is removed from this |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
557 # set. |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
558 self._activecommands = set() |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
559 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
560 def onframerecv(self, frame): |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
561 """Process a frame that has been received off the wire. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
562 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
563 Returns a dict with an ``action`` key that details what action, |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
564 if any, the consumer should take next. |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
565 """ |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
566 if not frame.streamid % 2: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
567 self._state = 'errored' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
568 return self._makeerrorresult( |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
569 _('received frame with even numbered stream ID: %d') % |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
570 frame.streamid) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
571 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
572 if frame.streamid not in self._incomingstreams: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
573 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
574 self._state = 'errored' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
575 return self._makeerrorresult( |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
576 _('received frame on unknown inactive stream without ' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
577 'beginning of stream flag set')) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
578 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
579 self._incomingstreams[frame.streamid] = stream(frame.streamid) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
580 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
581 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
582 # TODO handle decoding frames |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
583 self._state = 'errored' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
584 raise error.ProgrammingError('support for decoding stream payloads ' |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
585 'not yet implemented') |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
586 |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
587 if frame.streamflags & STREAM_FLAG_END_STREAM: |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
588 del self._incomingstreams[frame.streamid] |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
589 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
590 handlers = { |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
591 'idle': self._onframeidle, |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
592 'command-receiving': self._onframecommandreceiving, |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
593 'errored': self._onframeerrored, |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
594 } |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
595 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
596 meth = handlers.get(self._state) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
597 if not meth: |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
598 raise error.ProgrammingError('unhandled state: %s' % self._state) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
599 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
600 return meth(frame) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
601 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
602 def onbytesresponseready(self, stream, requestid, data): |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
603 """Signal that a bytes response is ready to be sent to the client. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
604 |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
605 The raw bytes response is passed as an argument. |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
606 """ |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
607 ensureserverstream(stream) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
608 |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
609 def sendframes(): |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
610 for frame in createbytesresponseframesfrombytes(stream, requestid, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
611 data): |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
612 yield frame |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
613 |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
614 self._activecommands.remove(requestid) |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
615 |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
616 result = sendframes() |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
617 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
618 if self._deferoutput: |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
619 self._bufferedframegens.append(result) |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
620 return 'noop', {} |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
621 else: |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
622 return 'sendframes', { |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
623 'framegen': result, |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
624 } |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
625 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
626 def oninputeof(self): |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
627 """Signals that end of input has been received. |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
628 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
629 No more frames will be received. All pending activity should be |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
630 completed. |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
631 """ |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
632 # TODO should we do anything about in-flight commands? |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
633 |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
634 if not self._deferoutput or not self._bufferedframegens: |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
635 return 'noop', {} |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
636 |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
637 # If we buffered all our responses, emit those. |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
638 def makegen(): |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
639 for gen in self._bufferedframegens: |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
640 for frame in gen: |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
641 yield frame |
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
642 |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
643 return 'sendframes', { |
37059
861e9d37e56e
wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37058
diff
changeset
|
644 'framegen': makegen(), |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
645 } |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
646 |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
647 def onapplicationerror(self, stream, requestid, msg): |
37288
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
648 ensureserverstream(stream) |
9bfcbe4f4745
wireproto: add streams to frame-based protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37287
diff
changeset
|
649 |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
650 return 'sendframes', { |
37287
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
651 'framegen': createerrorframe(stream, requestid, msg, |
3ed344546d9e
wireproto: start to associate frame generation with a stream
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37087
diff
changeset
|
652 application=True), |
37058
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
653 } |
61393f888dfe
wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37055
diff
changeset
|
654 |
37289
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
655 def makeoutputstream(self): |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
656 """Create a stream to be used for sending data to the client.""" |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
657 streamid = self._nextoutgoingstreamid |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
658 self._nextoutgoingstreamid += 2 |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
659 |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
660 s = stream(streamid) |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
661 self._outgoingstreams[streamid] = s |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
662 |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
663 return s |
5fadc63ac99f
wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37288
diff
changeset
|
664 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
665 def _makeerrorresult(self, msg): |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
666 return 'error', { |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
667 'message': msg, |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
668 } |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
669 |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
670 def _makeruncommandresult(self, requestid): |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
671 entry = self._receivingcommands[requestid] |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
672 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
673 if not entry['requestdone']: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
674 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
675 raise error.ProgrammingError('should not be called without ' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
676 'requestdone set') |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
677 |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
678 del self._receivingcommands[requestid] |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
679 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
680 if self._receivingcommands: |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
681 self._state = 'command-receiving' |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
682 else: |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
683 self._state = 'idle' |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
684 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
685 # Decode the payloads as CBOR. |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
686 entry['payload'].seek(0) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
687 request = cbor.load(entry['payload']) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
688 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
689 if b'name' not in request: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
690 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
691 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
692 _('command request missing "name" field')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
693 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
694 if b'args' not in request: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
695 request[b'args'] = {} |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
696 |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
697 assert requestid not in self._activecommands |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
698 self._activecommands.add(requestid) |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
699 |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
700 return 'runcommand', { |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
701 'requestid': requestid, |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
702 'command': request[b'name'], |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
703 'args': request[b'args'], |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
704 'data': entry['data'].getvalue() if entry['data'] else None, |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
705 } |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
706 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
707 def _makewantframeresult(self): |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
708 return 'wantframe', { |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
709 'state': self._state, |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
710 } |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
711 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
712 def _validatecommandrequestframe(self, frame): |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
713 new = frame.flags & FLAG_COMMAND_REQUEST_NEW |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
714 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
715 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
716 if new and continuation: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
717 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
718 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
719 _('received command request frame with both new and ' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
720 'continuation flags set')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
721 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
722 if not new and not continuation: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
723 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
724 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
725 _('received command request frame with neither new nor ' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
726 'continuation flags set')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
727 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
728 def _onframeidle(self, frame): |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
729 # The only frame type that should be received in this state is a |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
730 # command request. |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
731 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST: |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
732 self._state = 'errored' |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
733 return self._makeerrorresult( |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
734 _('expected command request frame; got %d') % frame.typeid) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
735 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
736 res = self._validatecommandrequestframe(frame) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
737 if res: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
738 return res |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
739 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
740 if frame.requestid in self._receivingcommands: |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
741 self._state = 'errored' |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
742 return self._makeerrorresult( |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
743 _('request with ID %d already received') % frame.requestid) |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
744 |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
745 if frame.requestid in self._activecommands: |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
746 self._state = 'errored' |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
747 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
748 _('request with ID %d is already active') % frame.requestid) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
749 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
750 new = frame.flags & FLAG_COMMAND_REQUEST_NEW |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
751 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
752 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
753 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
754 if not new: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
755 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
756 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
757 _('received command request frame without new flag set')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
758 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
759 payload = util.bytesio() |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
760 payload.write(frame.payload) |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
761 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
762 self._receivingcommands[frame.requestid] = { |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
763 'payload': payload, |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
764 'data': None, |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
765 'requestdone': not moreframes, |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
766 'expectingdata': bool(expectingdata), |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
767 } |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
768 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
769 # This is the final frame for this request. Dispatch it. |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
770 if not moreframes and not expectingdata: |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
771 return self._makeruncommandresult(frame.requestid) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
772 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
773 assert moreframes or expectingdata |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
774 self._state = 'command-receiving' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
775 return self._makewantframeresult() |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
776 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
777 def _onframecommandreceiving(self, frame): |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
778 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
779 # Process new command requests as such. |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
780 if frame.flags & FLAG_COMMAND_REQUEST_NEW: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
781 return self._onframeidle(frame) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
782 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
783 res = self._validatecommandrequestframe(frame) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
784 if res: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
785 return res |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
786 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
787 # All other frames should be related to a command that is currently |
37066
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
788 # receiving but is not active. |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
789 if frame.requestid in self._activecommands: |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
790 self._state = 'errored' |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
791 return self._makeerrorresult( |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
792 _('received frame for request that is still active: %d') % |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
793 frame.requestid) |
39304dd63589
wireproto: explicitly track which requests are active
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37065
diff
changeset
|
794 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
795 if frame.requestid not in self._receivingcommands: |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
796 self._state = 'errored' |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
797 return self._makeerrorresult( |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
798 _('received frame for request that is not receiving: %d') % |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
799 frame.requestid) |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
800 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
801 entry = self._receivingcommands[frame.requestid] |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
802 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
803 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
804 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
805 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
806 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
807 if entry['requestdone']: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
808 self._state = 'errored' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
809 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
810 _('received command request frame when request frames ' |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
811 'were supposedly done')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
812 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
813 if expectingdata != entry['expectingdata']: |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
814 self._state = 'errored' |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
815 return self._makeerrorresult( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
816 _('mismatch between expect data flag and previous frame')) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
817 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
818 entry['payload'].write(frame.payload) |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
819 |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
820 if not moreframes: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
821 entry['requestdone'] = True |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
822 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
823 if not moreframes and not expectingdata: |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
824 return self._makeruncommandresult(frame.requestid) |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
825 |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
826 return self._makewantframeresult() |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
827 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
828 elif frame.typeid == FRAME_TYPE_COMMAND_DATA: |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
829 if not entry['expectingdata']: |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
830 self._state = 'errored' |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
831 return self._makeerrorresult(_( |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
832 'received command data frame for request that is not ' |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
833 'expecting data: %d') % frame.requestid) |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
834 |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
835 if entry['data'] is None: |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
836 entry['data'] = util.bytesio() |
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
837 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
838 return self._handlecommanddataframe(frame, entry) |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
839 else: |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
840 self._state = 'errored' |
37292
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
841 return self._makeerrorresult(_( |
3d0e2cd86e05
wireproto: use CBOR for command requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37291
diff
changeset
|
842 'received unexpected frame type: %d') % frame.typeid) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
843 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
844 def _handlecommanddataframe(self, frame, entry): |
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
845 assert frame.typeid == FRAME_TYPE_COMMAND_DATA |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
846 |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
847 # TODO support streaming data instead of buffering it. |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
848 entry['data'].write(frame.payload) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
849 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
850 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION: |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
851 return self._makewantframeresult() |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
852 elif frame.flags & FLAG_COMMAND_DATA_EOS: |
37061
c5e9c3b47366
wireproto: support for receiving multiple requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37060
diff
changeset
|
853 entry['data'].seek(0) |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
854 return self._makeruncommandresult(frame.requestid) |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
855 else: |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
856 self._state = 'errored' |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
857 return self._makeerrorresult(_('command data frame without ' |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
858 'flags')) |
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
859 |
37064
884a0c1604ad
wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37063
diff
changeset
|
860 def _onframeerrored(self, frame): |
37055
8c3c47362934
wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37054
diff
changeset
|
861 return self._makeerrorresult(_('server already errored')) |