Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/wireprotoframing.py @ 37054:40206e227412
wireproto: define and implement protocol for issuing requests
The existing HTTP and SSH wire protocols suffer from a host of flaws
and shortcomings. I've been wanting to rewrite the protocol for a while
now. Supporting partial clone - which will require new wire protocol
commands and capabilities - and other advanced server functionality
will be much easier if we start from a clean slate and don't have
to be constrained by limitations of the existing wire protocol.
This commit starts to introduce a new data exchange format for
use over the wire protocol.
The new protocol is built on top of "frames," which are atomic
units of metadata + data. Frames will make it easier to implement
proxies and other mechanisms that want to inspect data without
having to maintain state. The existing frame metadata is very
minimal and it will evolve heavily. (We will eventually support
things like concurrent requests, out-of-order responses,
compression, side-channels for status updates, etc. Some of
these will require additions to the frame header.)
Another benefit of frames is that all reads are of a fixed size.
A reader works by consuming a frame header, extracting the payload
length, then reading that many bytes. No lookahead, buffering, or
memory reallocations are needed.
The new protocol attempts to be transport agnostic. I want all that's
required to use the new protocol to be a pair of unidirectional,
half-duplex pipes. (Yes, we will eventually make use of full-duplex
pipes, but that's for another commit.) Notably, when the SSH
transport switches to this new protocol, stderr will be unused.
This is by design: the lack of stderr on HTTP harms protocol
behavior there. By shoehorning everything into a pair of pipes,
we can have more consistent behavior across transports.
We currently only define the client side parts of the new protocol,
specifically the bits for requesting that a command run. This keeps
the new code and feature small and somewhat easy to review.
We add support to `hg debugwireproto` for writing frames into
HTTP request bodies. Our tests that issue commands to the new
HTTP endpoint have been updated to transmit frames. The server
bits haven't been touched to consume the frames yet. This will
occur in the next commit...
Astute readers may notice that the command name is transmitted in
both the HTTP request URL and the command request frame. This is
partially a kludge from me initially implementing the frame-based
protocol for SSH first. But it is also a feature: I intend to
eventually support issuing multiple commands per HTTP request. This
will allow us to replace the abomination that is the "batch" wire
protocol command with a protocol-level mechanism for performing
multi-dispatch. Because I want the frame-based protocol to be
as similar as possible across transports, I'd rather we (redundantly)
include the command name in the frame than differ behavior between
transports that have out-of-band routing information (like HTTP)
readily available.
Differential Revision: https://phab.mercurial-scm.org/D2851
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Mon, 19 Mar 2018 16:49:53 -0700 |
parents | |
children | 8c3c47362934 |
rev | line source |
---|---|
37054
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # wireprotoframing.py - unified framing protocol for wire protocol |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 # |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 # This file contains functionality to support the unified frame-based wire |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 # protocol. For details about the protocol, see |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
10 # `hg help internals.wireprotocol`. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
11 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
12 from __future__ import absolute_import |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
13 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
14 import struct |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
15 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
16 from . import ( |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
17 util, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
18 ) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
19 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
20 FRAME_HEADER_SIZE = 4 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
21 DEFAULT_MAX_FRAME_SIZE = 32768 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
22 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
23 FRAME_TYPE_COMMAND_NAME = 0x01 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
24 FRAME_TYPE_COMMAND_ARGUMENT = 0x02 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
25 FRAME_TYPE_COMMAND_DATA = 0x03 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
26 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
27 FRAME_TYPES = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
28 b'command-name': FRAME_TYPE_COMMAND_NAME, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
29 b'command-argument': FRAME_TYPE_COMMAND_ARGUMENT, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
30 b'command-data': FRAME_TYPE_COMMAND_DATA, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
31 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
32 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
33 FLAG_COMMAND_NAME_EOS = 0x01 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
34 FLAG_COMMAND_NAME_HAVE_ARGS = 0x02 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
35 FLAG_COMMAND_NAME_HAVE_DATA = 0x04 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
36 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
37 FLAGS_COMMAND = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
38 b'eos': FLAG_COMMAND_NAME_EOS, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
39 b'have-args': FLAG_COMMAND_NAME_HAVE_ARGS, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
40 b'have-data': FLAG_COMMAND_NAME_HAVE_DATA, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
41 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
42 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
43 FLAG_COMMAND_ARGUMENT_CONTINUATION = 0x01 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
44 FLAG_COMMAND_ARGUMENT_EOA = 0x02 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
45 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
46 FLAGS_COMMAND_ARGUMENT = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
47 b'continuation': FLAG_COMMAND_ARGUMENT_CONTINUATION, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
48 b'eoa': FLAG_COMMAND_ARGUMENT_EOA, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
49 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
50 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
51 FLAG_COMMAND_DATA_CONTINUATION = 0x01 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
52 FLAG_COMMAND_DATA_EOS = 0x02 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
53 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
54 FLAGS_COMMAND_DATA = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
55 b'continuation': FLAG_COMMAND_DATA_CONTINUATION, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
56 b'eos': FLAG_COMMAND_DATA_EOS, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
57 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
58 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
59 # Maps frame types to their available flags. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
60 FRAME_TYPE_FLAGS = { |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
61 FRAME_TYPE_COMMAND_NAME: FLAGS_COMMAND, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
62 FRAME_TYPE_COMMAND_ARGUMENT: FLAGS_COMMAND_ARGUMENT, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
63 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA, |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
64 } |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
65 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
66 ARGUMENT_FRAME_HEADER = struct.Struct(r'<HH') |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
67 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
68 def makeframe(frametype, frameflags, payload): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
69 """Assemble a frame into a byte array.""" |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
70 # TODO assert size of payload. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
71 frame = bytearray(FRAME_HEADER_SIZE + len(payload)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
72 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
73 l = struct.pack(r'<I', len(payload)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
74 frame[0:3] = l[0:3] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
75 frame[3] = (frametype << 4) | frameflags |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
76 frame[4:] = payload |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
77 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
78 return frame |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
79 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
80 def makeframefromhumanstring(s): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
81 """Given a string of the form: <type> <flags> <payload>, creates a frame. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
82 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
83 This can be used by user-facing applications and tests for creating |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
84 frames easily without having to type out a bunch of constants. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
85 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
86 Frame type and flags can be specified by integer or named constant. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
87 Flags can be delimited by `|` to bitwise OR them together. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
88 """ |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
89 frametype, frameflags, payload = s.split(b' ', 2) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
90 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
91 if frametype in FRAME_TYPES: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
92 frametype = FRAME_TYPES[frametype] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
93 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
94 frametype = int(frametype) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
95 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
96 finalflags = 0 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
97 validflags = FRAME_TYPE_FLAGS[frametype] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
98 for flag in frameflags.split(b'|'): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
99 if flag in validflags: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
100 finalflags |= validflags[flag] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
101 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
102 finalflags |= int(flag) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
103 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
104 payload = util.unescapestr(payload) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
105 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
106 return makeframe(frametype, finalflags, payload) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
107 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
108 def createcommandframes(cmd, args, datafh=None): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
109 """Create frames necessary to transmit a request to run a command. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
110 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
111 This is a generator of bytearrays. Each item represents a frame |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
112 ready to be sent over the wire to a peer. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
113 """ |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
114 flags = 0 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
115 if args: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
116 flags |= FLAG_COMMAND_NAME_HAVE_ARGS |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
117 if datafh: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
118 flags |= FLAG_COMMAND_NAME_HAVE_DATA |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
119 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
120 if not flags: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
121 flags |= FLAG_COMMAND_NAME_EOS |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
122 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
123 yield makeframe(FRAME_TYPE_COMMAND_NAME, flags, cmd) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
124 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
125 for i, k in enumerate(sorted(args)): |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
126 v = args[k] |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
127 last = i == len(args) - 1 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
128 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
129 # TODO handle splitting of argument values across frames. |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
130 payload = bytearray(ARGUMENT_FRAME_HEADER.size + len(k) + len(v)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
131 offset = 0 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
132 ARGUMENT_FRAME_HEADER.pack_into(payload, offset, len(k), len(v)) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
133 offset += ARGUMENT_FRAME_HEADER.size |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
134 payload[offset:offset + len(k)] = k |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
135 offset += len(k) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
136 payload[offset:offset + len(v)] = v |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
137 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
138 flags = FLAG_COMMAND_ARGUMENT_EOA if last else 0 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
139 yield makeframe(FRAME_TYPE_COMMAND_ARGUMENT, flags, payload) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
140 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
141 if datafh: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
142 while True: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
143 data = datafh.read(DEFAULT_MAX_FRAME_SIZE) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
144 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
145 done = False |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
146 if len(data) == DEFAULT_MAX_FRAME_SIZE: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
147 flags = FLAG_COMMAND_DATA_CONTINUATION |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
148 else: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
149 flags = FLAG_COMMAND_DATA_EOS |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
150 assert datafh.read(1) == b'' |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
151 done = True |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
152 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
153 yield makeframe(FRAME_TYPE_COMMAND_DATA, flags, data) |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
154 |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
155 if done: |
40206e227412
wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
156 break |