annotate mercurial/wireprotoframing.py @ 37055:8c3c47362934

wireproto: implement basic frame reading and processing We just implemented support for writing frames. Now let's implement support for reading them. The bulk of the new code is for a class that maintains the state of a server. Essentially, you construct an instance, feed frames to it, and it tells you what you should do next. The design is inspired by the "sans I/O" movement and the reactor pattern. We don't want to perform I/O or any major blocking event during frame ingestion because this arbitrarily limits ways that server pieces can be implemented. For example, it makes it much harder to swap in an alternate implementation based on asyncio or do crazy things like have requests dispatch to other processes. We do still implement readframe() which does I/O. But it is decoupled from the server reactor. And important parsing of frame headers is a standalone function. So I/O is only needed to obtain frame data. Because testing server-side ingest is useful and difficult on running servers, we create a new "debugreflect" endpoint that will echo back to the client what was received and how it was interpreted. This could be useful for a server admin, someone implementing a client. But immediately, it is useful for testing: we're able to demonstrate that frames are parsed correctly and turned into requests to run commands without having to implement command dispatch on the server! In addition, we implement Python level unit tests for the reactor. This is vastly more efficient than sending requests to the "debugreflect" endpoint and vastly more powerful for advanced testing. Differential Revision: https://phab.mercurial-scm.org/D2852
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 14 Mar 2018 15:25:06 -0700
parents 40206e227412
children 61393f888dfe
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 import struct
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
16 from .i18n import _
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 from . import (
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
18 error,
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 util,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 )
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 FRAME_HEADER_SIZE = 4
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25 FRAME_TYPE_COMMAND_NAME = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 FRAME_TYPE_COMMAND_ARGUMENT = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 FRAME_TYPE_COMMAND_DATA = 0x03
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 FRAME_TYPES = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 b'command-name': FRAME_TYPE_COMMAND_NAME,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31 b'command-argument': FRAME_TYPE_COMMAND_ARGUMENT,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 b'command-data': FRAME_TYPE_COMMAND_DATA,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 FLAG_COMMAND_NAME_EOS = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 FLAG_COMMAND_NAME_HAVE_ARGS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37 FLAG_COMMAND_NAME_HAVE_DATA = 0x04
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 FLAGS_COMMAND = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40 b'eos': FLAG_COMMAND_NAME_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 b'have-args': FLAG_COMMAND_NAME_HAVE_ARGS,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 b'have-data': FLAG_COMMAND_NAME_HAVE_DATA,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 FLAG_COMMAND_ARGUMENT_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46 FLAG_COMMAND_ARGUMENT_EOA = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 FLAGS_COMMAND_ARGUMENT = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 b'continuation': FLAG_COMMAND_ARGUMENT_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 b'eoa': FLAG_COMMAND_ARGUMENT_EOA,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
51 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
54 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
55
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
56 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
57 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
58 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62 FRAME_TYPE_FLAGS = {
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63 FRAME_TYPE_COMMAND_NAME: FLAGS_COMMAND,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64 FRAME_TYPE_COMMAND_ARGUMENT: FLAGS_COMMAND_ARGUMENT,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66 }
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
67
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 ARGUMENT_FRAME_HEADER = struct.Struct(r'<HH')
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
69
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 def makeframe(frametype, frameflags, payload):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 frame[0:3] = l[0:3]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 frame[3] = (frametype << 4) | frameflags
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 frame[4:] = payload
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
80 return frame
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
81
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82 def makeframefromhumanstring(s):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
83 """Given a string of the form: <type> <flags> <payload>, creates a frame.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
86 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
87
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88 Frame type and flags can be specified by integer or named constant.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89 Flags can be delimited by `|` to bitwise OR them together.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90 """
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 frametype, frameflags, payload = s.split(b' ', 2)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
92
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
93 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
94 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
95 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
96 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
97
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
100 for flag in frameflags.split(b'|'):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
101 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102 finalflags |= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
104 finalflags |= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
105
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
106 payload = util.unescapestr(payload)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
107
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
108 return makeframe(frametype, finalflags, payload)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
109
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
110 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
111 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
112
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
113 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
114 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
115 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
116 # 24 bits payload length (little endian)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
117 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
118 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
119 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
120 framelength = data[0] + 256 * data[1] + 16384 * data[2]
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
121 typeflags = data[3]
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
122
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
123 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
124 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
125
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
126 return frametype, frameflags, framelength
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
127
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
128 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
129 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
130
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
131 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
132 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
133 seen.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
134 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
135 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
136
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
137 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
138
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
139 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
140 return None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
141
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
142 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
143 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
144 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
145
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
146 frametype, frameflags, framelength = parseheader(header)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
147
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
148 payload = fh.read(framelength)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
149 if len(payload) != framelength:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
150 raise error.Abort(_('frame length error: expected %d; got %d') %
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
151 (framelength, len(payload)))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
152
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
153 return frametype, frameflags, payload
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
154
37054
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
155 def createcommandframes(cmd, args, datafh=None):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
156 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
157
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
158 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
159 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
160 """
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
161 flags = 0
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
162 if args:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
163 flags |= FLAG_COMMAND_NAME_HAVE_ARGS
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
164 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
165 flags |= FLAG_COMMAND_NAME_HAVE_DATA
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
166
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 if not flags:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168 flags |= FLAG_COMMAND_NAME_EOS
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
169
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
170 yield makeframe(FRAME_TYPE_COMMAND_NAME, flags, cmd)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
171
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
172 for i, k in enumerate(sorted(args)):
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173 v = args[k]
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174 last = i == len(args) - 1
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
176 # TODO handle splitting of argument values across frames.
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
177 payload = bytearray(ARGUMENT_FRAME_HEADER.size + len(k) + len(v))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
178 offset = 0
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
179 ARGUMENT_FRAME_HEADER.pack_into(payload, offset, len(k), len(v))
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
180 offset += ARGUMENT_FRAME_HEADER.size
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
181 payload[offset:offset + len(k)] = k
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
182 offset += len(k)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
183 payload[offset:offset + len(v)] = v
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
184
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
185 flags = FLAG_COMMAND_ARGUMENT_EOA if last else 0
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
186 yield makeframe(FRAME_TYPE_COMMAND_ARGUMENT, flags, payload)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
187
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
188 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
189 while True:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
190 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
191
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
192 done = False
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
193 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
194 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
195 else:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
196 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
197 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
198 done = True
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
199
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
200 yield makeframe(FRAME_TYPE_COMMAND_DATA, flags, data)
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
201
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
202 if done:
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
203 break
37055
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
204
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
205 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
206 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
207
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
208 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
209 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
210 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
211 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
212
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
213 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
214 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
215 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
216 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
217 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
218 *run a command*), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
219
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
220 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
221 this class.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
222
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
223 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
224 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
225 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
226 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
227 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
228 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
229 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
230
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
231 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
232
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
233 error
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
234 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
235
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
236 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
237 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
238 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
239
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
240 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
241 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
242 more frames from the client before anything interesting can be done.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
243 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
244
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
245 def __init__(self):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
246 self._state = 'idle'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
247 self._activecommand = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
248 self._activeargs = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
249 self._activedata = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
250 self._expectingargs = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
251 self._expectingdata = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
252 self._activeargname = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
253 self._activeargchunks = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
254
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
255 def onframerecv(self, frametype, frameflags, payload):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
256 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
257
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
258 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
259 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
260 """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
261 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
262 'idle': self._onframeidle,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
263 'command-receiving-args': self._onframereceivingargs,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
264 'command-receiving-data': self._onframereceivingdata,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
265 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
266 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
267
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
268 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
269 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
270 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
271
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
272 return meth(frametype, frameflags, payload)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
273
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
274 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
275 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
276 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
277 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
278
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
279 def _makeruncommandresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
280 return 'runcommand', {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
281 'command': self._activecommand,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
282 'args': self._activeargs,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
283 'data': self._activedata.getvalue() if self._activedata else None,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
284 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
285
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
286 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
287 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
288 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
289 }
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
290
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
291 def _onframeidle(self, frametype, frameflags, payload):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
292 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
293 # command request.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
294 if frametype != FRAME_TYPE_COMMAND_NAME:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
295 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
296 return self._makeerrorresult(
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
297 _('expected command frame; got %d') % frametype)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
298
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
299 self._activecommand = payload
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
300 self._activeargs = {}
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
301 self._activedata = None
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
302
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
303 if frameflags & FLAG_COMMAND_NAME_EOS:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
304 return self._makeruncommandresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
305
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
306 self._expectingargs = bool(frameflags & FLAG_COMMAND_NAME_HAVE_ARGS)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
307 self._expectingdata = bool(frameflags & FLAG_COMMAND_NAME_HAVE_DATA)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
308
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
309 if self._expectingargs:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
310 self._state = 'command-receiving-args'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
311 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
312 elif self._expectingdata:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
313 self._activedata = util.bytesio()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
314 self._state = 'command-receiving-data'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
315 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
316 else:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
317 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
318 return self._makeerrorresult(_('missing frame flags on '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
319 'command frame'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
320
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
321 def _onframereceivingargs(self, frametype, frameflags, payload):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
322 if frametype != FRAME_TYPE_COMMAND_ARGUMENT:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
323 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
324 return self._makeerrorresult(_('expected command argument '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
325 'frame; got %d') % frametype)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
326
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
327 offset = 0
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
328 namesize, valuesize = ARGUMENT_FRAME_HEADER.unpack_from(payload)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
329 offset += ARGUMENT_FRAME_HEADER.size
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
330
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
331 # The argument name MUST fit inside the frame.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
332 argname = bytes(payload[offset:offset + namesize])
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
333 offset += namesize
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
334
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
335 if len(argname) != namesize:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
336 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
337 return self._makeerrorresult(_('malformed argument frame: '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
338 'partial argument name'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
339
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
340 argvalue = bytes(payload[offset:])
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
341
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
342 # Argument value spans multiple frames. Record our active state
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
343 # and wait for the next frame.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
344 if frameflags & FLAG_COMMAND_ARGUMENT_CONTINUATION:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
345 raise error.ProgrammingError('not yet implemented')
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
346 self._activeargname = argname
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
347 self._activeargchunks = [argvalue]
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
348 self._state = 'command-arg-continuation'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
349 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
350
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
351 # Common case: the argument value is completely contained in this
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
352 # frame.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
353
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
354 if len(argvalue) != valuesize:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
355 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
356 return self._makeerrorresult(_('malformed argument frame: '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
357 'partial argument value'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
358
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
359 self._activeargs[argname] = argvalue
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
360
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
361 if frameflags & FLAG_COMMAND_ARGUMENT_EOA:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
362 if self._expectingdata:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
363 self._state = 'command-receiving-data'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
364 self._activedata = util.bytesio()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
365 # TODO signal request to run a command once we don't
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
366 # buffer data frames.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
367 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
368 else:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
369 self._state = 'waiting'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
370 return self._makeruncommandresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
371 else:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
372 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
373
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
374 def _onframereceivingdata(self, frametype, frameflags, payload):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
375 if frametype != FRAME_TYPE_COMMAND_DATA:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
376 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
377 return self._makeerrorresult(_('expected command data frame; '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
378 'got %d') % frametype)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
379
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
380 # TODO support streaming data instead of buffering it.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
381 self._activedata.write(payload)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
382
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
383 if frameflags & FLAG_COMMAND_DATA_CONTINUATION:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
384 return self._makewantframeresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
385 elif frameflags & FLAG_COMMAND_DATA_EOS:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
386 self._activedata.seek(0)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
387 self._state = 'idle'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
388 return self._makeruncommandresult()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
389 else:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
390 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
391 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
392 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
393
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
394 def _onframeerrored(self, frametype, frameflags, payload):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
395 return self._makeerrorresult(_('server already errored'))