Mercurial > public > mercurial-scm > hg

annotate mercurial/utils/cborutil.py @ 48875:6000f5b25c9b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
py2: remove simple from __future__ statements These were needed for Python 2 support. Now that our linter no longer mandates these, we can start deleting them. Differential Revision: https://phab.mercurial-scm.org/D12254
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 03 Mar 2022 17:34:00 +0100
parents 89a2afe31e82
children f254fc73d956
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # cborutil.py - CBOR extensions
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 import struct
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
10 import sys
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11
39456
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
12 from .. import pycompat
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
13
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 # Very short very of RFC 7049...
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 # Each item begins with a byte. The 3 high bits of that byte denote the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 # "major type." The lower 5 bits denote the "subtype." Each major type
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 # has its own encoding mechanism.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 # Most types have lengths. However, bytestring, string, array, and map
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 # can be indefinite length. These are denotes by a subtype with value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 # Sub-components of those types then come afterwards and are terminated
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 # by a "break" byte.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25 MAJOR_TYPE_UINT = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 MAJOR_TYPE_NEGINT = 1
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 MAJOR_TYPE_BYTESTRING = 2
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 MAJOR_TYPE_STRING = 3
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 MAJOR_TYPE_ARRAY = 4
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 MAJOR_TYPE_MAP = 5
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31 MAJOR_TYPE_SEMANTIC = 6
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 MAJOR_TYPE_SPECIAL = 7
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34 SUBTYPE_MASK = 0b00011111
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
36 SUBTYPE_FALSE = 20
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
37 SUBTYPE_TRUE = 21
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
38 SUBTYPE_NULL = 22
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 SUBTYPE_HALF_FLOAT = 25
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40 SUBTYPE_SINGLE_FLOAT = 26
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 SUBTYPE_DOUBLE_FLOAT = 27
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 SUBTYPE_INDEFINITE = 31
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
44 SEMANTIC_TAG_FINITE_SET = 258
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
45
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46 # Indefinite types begin with their major type ORd with information value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47 BEGIN_INDEFINITE_BYTESTRING = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
48 '>B', MAJOR_TYPE_BYTESTRING << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
49 )
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 BEGIN_INDEFINITE_ARRAY = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
51 '>B', MAJOR_TYPE_ARRAY << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
52 )
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 BEGIN_INDEFINITE_MAP = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
54 '>B', MAJOR_TYPE_MAP << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
55 )
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
56
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
57 ENCODED_LENGTH_1 = struct.Struct('>B')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
58 ENCODED_LENGTH_2 = struct.Struct('>BB')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
59 ENCODED_LENGTH_3 = struct.Struct('>BH')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
60 ENCODED_LENGTH_4 = struct.Struct('>BL')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
61 ENCODED_LENGTH_5 = struct.Struct('>BQ')
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63 # The break ends an indefinite length item.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64 BREAK = b'\xff'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 BREAK_INT = 255
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
67
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 def encodelength(majortype, length):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
69 """Obtain a value encoding the major type and its length."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 if length < 24:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71 return ENCODED_LENGTH_1.pack(majortype << 5 | length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 elif length < 256:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 return ENCODED_LENGTH_2.pack(majortype << 5 | 24, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74 elif length < 65536:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 return ENCODED_LENGTH_3.pack(majortype << 5 | 25, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 elif length < 4294967296:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 return ENCODED_LENGTH_4.pack(majortype << 5 | 26, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79 return ENCODED_LENGTH_5.pack(majortype << 5 | 27, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
80
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
81
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82 def streamencodebytestring(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
83 yield encodelength(MAJOR_TYPE_BYTESTRING, len(v))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84 yield v
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
86
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
87 def streamencodebytestringfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88 """Convert an iterator of chunks to an indefinite bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90 Given an input that is iterable and each element in the iterator is
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 representable as bytes, emit an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
92 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
93 yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
94
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
95 for chunk in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
96 yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
97 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
100
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
101
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102 def streamencodeindefinitebytestring(source, chunksize=65536):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 """Given a large source buffer, emit as an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
104
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
105 This is a generator of chunks constituting the encoded CBOR data.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
106 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
107 yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
108
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
109 i = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
110 l = len(source)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
111
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
112 while True:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
113 chunk = source[i : i + chunksize]
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
114 i += len(chunk)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
115
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
116 yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
117 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
118
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
119 if i >= l:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
120 break
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
121
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
122 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
123
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
124
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
125 def streamencodeint(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
126 if v >= 18446744073709551616 or v < -18446744073709551616:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
127 raise ValueError(b'big integers not supported')
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
128
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
129 if v >= 0:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
130 yield encodelength(MAJOR_TYPE_UINT, v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
131 else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
132 yield encodelength(MAJOR_TYPE_NEGINT, abs(v) - 1)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
133
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
134
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
135 def streamencodearray(l):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
136 """Encode a known size iterable to an array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
137
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
138 yield encodelength(MAJOR_TYPE_ARRAY, len(l))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
139
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
140 for i in l:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
141 for chunk in streamencode(i):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
142 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
143
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
144
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
145 def streamencodearrayfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
146 """Encode an iterator of items to an indefinite length array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
147
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
148 yield BEGIN_INDEFINITE_ARRAY
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
149
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
150 for i in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
151 for chunk in streamencode(i):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
152 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
153
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
154 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
155
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
156
37898
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
157 def _mixedtypesortkey(v):
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
158 return type(v).__name__, v
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
159
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
160
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
161 def streamencodeset(s):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
162 # https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml defines
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
163 # semantic tag 258 for finite sets.
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
164 yield encodelength(MAJOR_TYPE_SEMANTIC, SEMANTIC_TAG_FINITE_SET)
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
165
37898
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
166 for chunk in streamencodearray(sorted(s, key=_mixedtypesortkey)):
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
169
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
170 def streamencodemap(d):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
171 """Encode dictionary to a generator.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
172
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173 Does not supporting indefinite length dictionaries.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175 yield encodelength(MAJOR_TYPE_MAP, len(d))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
176
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
177 for key, value in sorted(
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
178 pycompat.iteritems(d), key=lambda x: _mixedtypesortkey(x[0])
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
179 ):
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
180 for chunk in streamencode(key):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
181 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
182 for chunk in streamencode(value):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
183 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
184
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
185
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
186 def streamencodemapfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
187 """Given an iterable of (key, value), encode to an indefinite length map."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
188 yield BEGIN_INDEFINITE_MAP
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
189
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
190 for key, value in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
191 for chunk in streamencode(key):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
192 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
193 for chunk in streamencode(value):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
194 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
195
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
196 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
197
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
198
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
199 def streamencodebool(b):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
200 # major type 7, simple value 20 and 21.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
201 yield b'\xf5' if b else b'\xf4'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
202
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
203
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
204 def streamencodenone(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
205 # major type 7, simple value 22.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
206 yield b'\xf6'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
207
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
208
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
209 STREAM_ENCODERS = {
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
210 bytes: streamencodebytestring,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
211 int: streamencodeint,
39456
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
212 pycompat.long: streamencodeint,
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
213 list: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
214 tuple: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
215 dict: streamencodemap,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
216 set: streamencodeset,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
217 bool: streamencodebool,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
218 type(None): streamencodenone,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
219 }
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
220
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
221
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
222 def streamencode(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
223 """Encode a value in a streaming manner.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
224
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
225 Given an input object, encode it to CBOR recursively.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
226
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
227 Returns a generator of CBOR encoded bytes. There is no guarantee
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
228 that each emitted chunk fully decodes to a value or sub-value.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
229
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
230 Encoding is deterministic - unordered collections are sorted.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
231 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
232 fn = STREAM_ENCODERS.get(v.__class__)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
233
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
234 if not fn:
42480
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
235 # handle subtypes such as encoding.localstr and util.sortdict
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
236 for ty in STREAM_ENCODERS:
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
237 if not isinstance(v, ty):
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
238 continue
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
239 fn = STREAM_ENCODERS[ty]
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
240 break
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
241
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
242 if not fn:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
243 raise ValueError(b'do not know how to encode %s' % type(v))
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
244
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
245 return fn(v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
246
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
247
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
248 class CBORDecodeError(Exception):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
249 """Represents an error decoding CBOR."""
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
250
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
251
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
252 if sys.version_info.major >= 3:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
253
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
254 def _elementtointeger(b, i):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
255 return b[i]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
256
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
257
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
258 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
259
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
260 def _elementtointeger(b, i):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
261 return ord(b[i])
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
262
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
263
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
264 STRUCT_BIG_UBYTE = struct.Struct('>B')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
265 STRUCT_BIG_USHORT = struct.Struct(b'>H')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
266 STRUCT_BIG_ULONG = struct.Struct(b'>L')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
267 STRUCT_BIG_ULONGLONG = struct.Struct(b'>Q')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
268
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
269 SPECIAL_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
270 SPECIAL_START_INDEFINITE_BYTESTRING = 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
271 SPECIAL_START_ARRAY = 2
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
272 SPECIAL_START_MAP = 3
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
273 SPECIAL_START_SET = 4
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
274 SPECIAL_INDEFINITE_BREAK = 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
275
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
276
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
277 def decodeitem(b, offset=0):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
278 """Decode a new CBOR value from a buffer at offset.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
279
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
280 This function attempts to decode up to one complete CBOR value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
281 from ``b`` starting at offset ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
282
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
283 The beginning of a collection (such as an array, map, set, or
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
284 indefinite length bytestring) counts as a single value. For these
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
285 special cases, a state flag will indicate that a special value was seen.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
286
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
287 When called, the function either returns a decoded value or gives
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
288 a hint as to how many more bytes are needed to do so. By calling
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
289 the function repeatedly given a stream of bytes, the caller can
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
290 build up the original values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
291
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
292 Returns a tuple with the following elements:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
293
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
294 * Bool indicating whether a complete value was decoded.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
295 * A decoded value if first value is True otherwise None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
296 * Integer number of bytes. If positive, the number of bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
297 read. If negative, the number of bytes we need to read to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
298 decode this value or the next chunk in this value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
299 * One of the ``SPECIAL_*`` constants indicating special treatment
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
300 for this value. ``SPECIAL_NONE`` means this is a fully decoded
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
301 simple value (such as an integer or bool).
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
302 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
303
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
304 initial = _elementtointeger(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
305 offset += 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
306
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
307 majortype = initial >> 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
308 subtype = initial & SUBTYPE_MASK
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
309
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
310 if majortype == MAJOR_TYPE_UINT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
311 complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
312
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
313 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
314 return True, value, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
315 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
316 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
317
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
318 elif majortype == MAJOR_TYPE_NEGINT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
319 # Negative integers are the same as UINT except inverted minus 1.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
320 complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
321
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
322 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
323 return True, -value - 1, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
324 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
325 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
326
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
327 elif majortype == MAJOR_TYPE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
328 # Beginning of bytestrings are treated as uints in order to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
329 # decode their length, which may be indefinite.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
330 complete, size, readcount = decodeuint(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
331 subtype, b, offset, allowindefinite=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
332 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
333
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
334 # We don't know the size of the bytestring. It must be a definitive
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
335 # length since the indefinite subtype would be encoded in the initial
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
336 # byte.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
337 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
338 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
339
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
340 # We know the length of the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
341 if size is not None:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
342 # And the data is available in the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
343 if offset + readcount + size <= len(b):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
344 value = b[offset + readcount : offset + readcount + size]
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
345 return True, value, readcount + size + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
346
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
347 # And we need more data in order to return the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
348 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
349 wanted = len(b) - offset - readcount - size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
350 return False, None, wanted, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
351
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
352 # It is an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
353 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
354 return True, None, 1, SPECIAL_START_INDEFINITE_BYTESTRING
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
355
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
356 elif majortype == MAJOR_TYPE_STRING:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
357 raise CBORDecodeError(b'string major type not supported')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
358
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
359 elif majortype == MAJOR_TYPE_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
360 # Beginning of arrays are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
361 # length. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
362 complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
363
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
364 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
365 return True, size, readcount + 1, SPECIAL_START_ARRAY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
366 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
367 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
368
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
369 elif majortype == MAJOR_TYPE_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
370 # Beginning of maps are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
371 # number of elements. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
372 complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
373
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
374 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
375 return True, size, readcount + 1, SPECIAL_START_MAP
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
376 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
377 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
378
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
379 elif majortype == MAJOR_TYPE_SEMANTIC:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
380 # Semantic tag value is read the same as a uint.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
381 complete, tagvalue, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
382
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
383 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
384 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
385
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
386 # This behavior here is a little wonky. The main type being "decorated"
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
387 # by this semantic tag follows. A more robust parser would probably emit
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
388 # a special flag indicating this as a semantic tag and let the caller
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
389 # deal with the types that follow. But since we don't support many
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
390 # semantic tags, it is easier to deal with the special cases here and
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
391 # hide complexity from the caller. If we add support for more semantic
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
392 # tags, we should probably move semantic tag handling into the caller.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
393 if tagvalue == SEMANTIC_TAG_FINITE_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
394 if offset + readcount >= len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
395 return False, None, -1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
396
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
397 complete, size, readcount2, special = decodeitem(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
398 b, offset + readcount
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
399 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
400
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
401 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
402 return False, None, readcount2, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
403
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
404 if special != SPECIAL_START_ARRAY:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
405 raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
406 b'expected array after finite set semantic tag'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
407 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
408
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
409 return True, size, readcount + readcount2 + 1, SPECIAL_START_SET
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
410
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
411 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 raise CBORDecodeError(b'semantic tag %d not allowed' % tagvalue)
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
413
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
414 elif majortype == MAJOR_TYPE_SPECIAL:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
415 # Only specific values for the information field are allowed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
416 if subtype == SUBTYPE_FALSE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
417 return True, False, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
418 elif subtype == SUBTYPE_TRUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
419 return True, True, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
420 elif subtype == SUBTYPE_NULL:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
421 return True, None, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
422 elif subtype == SUBTYPE_INDEFINITE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
423 return True, None, 1, SPECIAL_INDEFINITE_BREAK
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
424 # If value is 24, subtype is in next byte.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
425 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
426 raise CBORDecodeError(b'special type %d not allowed' % subtype)
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
427 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
428 assert False
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
429
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
430
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
431 def decodeuint(subtype, b, offset=0, allowindefinite=False):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
432 """Decode an unsigned integer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
433
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
434 ``subtype`` is the lower 5 bits from the initial byte CBOR item
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
435 "header." ``b`` is a buffer containing bytes. ``offset`` points to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
436 the index of the first byte after the byte that ``subtype`` was
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
437 derived from.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
438
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
439 ``allowindefinite`` allows the special indefinite length value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
440 indicator.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
441
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
442 Returns a 3-tuple of (successful, value, count).
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
443
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
444 The first element is a bool indicating if decoding completed. The 2nd
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
445 is the decoded integer value or None if not fully decoded or the subtype
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
446 is 31 and ``allowindefinite`` is True. The 3rd value is the count of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
447 If positive, it is the number of additional bytes decoded. If negative,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
448 it is the number of additional bytes needed to decode this value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
449 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
450
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
451 # Small values are inline.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
452 if subtype < 24:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
453 return True, subtype, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
454 # Indefinite length specifier.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
455 elif subtype == 31:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
456 if allowindefinite:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
457 return True, None, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
458 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
459 raise CBORDecodeError(b'indefinite length uint not allowed here')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
460 elif subtype >= 28:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
461 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
462 b'unsupported subtype on integer type: %d' % subtype
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
463 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
464
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
465 if subtype == 24:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
466 s = STRUCT_BIG_UBYTE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
467 elif subtype == 25:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
468 s = STRUCT_BIG_USHORT
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
469 elif subtype == 26:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
470 s = STRUCT_BIG_ULONG
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
471 elif subtype == 27:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
472 s = STRUCT_BIG_ULONGLONG
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
473 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
474 raise CBORDecodeError(b'bounds condition checking violation')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
475
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
476 if len(b) - offset >= s.size:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
477 return True, s.unpack_from(b, offset)[0], s.size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
478 else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
479 return False, None, len(b) - offset - s.size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
480
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
481
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
482 class bytestringchunk(bytes):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
483 """Represents a chunk/segment in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
484
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
485 This behaves like a ``bytes`` but in addition has the ``isfirst``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
486 and ``islast`` attributes indicating whether this chunk is the first
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
487 or last in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
488 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
489
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
490 def __new__(cls, v, first=False, last=False):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
491 self = bytes.__new__(cls, v)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
492 self.isfirst = first
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
493 self.islast = last
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
494
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
495 return self
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
496
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
497
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
498 class sansiodecoder(object):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
499 """A CBOR decoder that doesn't perform its own I/O.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
500
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
501 To use, construct an instance and feed it segments containing
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
502 CBOR-encoded bytes via ``decode()``. The return value from ``decode()``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
503 indicates whether a fully-decoded value is available, how many bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
504 were consumed, and offers a hint as to how many bytes should be fed
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
505 in next time to decode the next value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
506
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
507 The decoder assumes it will decode N discrete CBOR values, not just
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
508 a single value. i.e. if the bytestream contains uints packed one after
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
509 the other, the decoder will decode them all, rather than just the initial
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
510 one.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
511
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
512 When ``decode()`` indicates a value is available, call ``getavailable()``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
513 to return all fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
514
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
515 ``decode()`` can partially decode input. It is up to the caller to keep
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
516 track of what data was consumed and to pass unconsumed data in on the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
517 next invocation.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
518
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
519 The decoder decodes atomically at the *item* level. See ``decodeitem()``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
520 If an *item* cannot be fully decoded, the decoder won't record it as
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
521 partially consumed. Instead, the caller will be instructed to pass in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
522 the initial bytes of this item on the next invocation. This does result
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
523 in some redundant parsing. But the overhead should be minimal.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
524
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
525 This decoder only supports a subset of CBOR as required by Mercurial.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
526 It lacks support for:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
527
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
528 * Indefinite length arrays
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
529 * Indefinite length maps
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
530 * Use of indefinite length bytestrings as keys or values within
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
531 arrays, maps, or sets.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
532 * Nested arrays, maps, or sets within sets
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
533 * Any semantic tag that isn't a mathematical finite set
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
534 * Floating point numbers
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
535 * Undefined special value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
536
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
537 CBOR types are decoded to Python types as follows:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
538
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
539 uint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
540 negint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
541 bytestring -> bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
542 map -> dict
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
543 array -> list
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
544 True -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
545 False -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
546 null -> None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
547 indefinite length bytestring chunk -> [bytestringchunk]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
548
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
549 The only non-obvious mapping here is an indefinite length bytestring
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
550 to the ``bytestringchunk`` type. This is to facilitate streaming
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
551 indefinite length bytestrings out of the decoder and to differentiate
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
552 a regular bytestring from an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
553 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
554
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
555 _STATE_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
556 _STATE_WANT_MAP_KEY = 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
557 _STATE_WANT_MAP_VALUE = 2
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
558 _STATE_WANT_ARRAY_VALUE = 3
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
559 _STATE_WANT_SET_VALUE = 4
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
560 _STATE_WANT_BYTESTRING_CHUNK_FIRST = 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
561 _STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT = 6
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
562
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
563 def __init__(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
564 # TODO add support for limiting size of bytestrings
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
565 # TODO add support for limiting number of keys / values in collections
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
566 # TODO add support for limiting size of buffered partial values
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
567
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
568 self.decodedbytecount = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
569
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
570 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
571
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
572 # Stack of active nested collections. Each entry is a dict describing
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
573 # the collection.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
574 self._collectionstack = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
575
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
576 # Fully decoded key to use for the current map.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
577 self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
578
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
579 # Fully decoded values available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
580 self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
581
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
582 @property
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
583 def inprogress(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
584 """Whether the decoder has partially decoded a value."""
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
585 return self._state != self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
586
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
587 def decode(self, b, offset=0):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
588 """Attempt to decode bytes from an input buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
589
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
590 ``b`` is a collection of bytes and ``offset`` is the byte
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
591 offset within that buffer from which to begin reading data.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
592
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
593 ``b`` must support ``len()`` and accessing bytes slices via
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
594 ``__slice__``. Typically ``bytes`` instances are used.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
595
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
596 Returns a tuple with the following fields:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
597
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
598 * Bool indicating whether values are available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
599 * Integer indicating the number of bytes that were fully consumed,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
600 starting from ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
601 * Integer indicating the number of bytes that are desired for the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
602 next call in order to decode an item.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
603 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
604 if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
605 return bool(self._decodedvalues), 0, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
606
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
607 initialoffset = offset
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
608
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
609 # We could easily split the body of this loop into a function. But
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
610 # Python performance is sensitive to function calls and collections
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
611 # are composed of many items. So leaving as a while loop could help
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
612 # with performance. One thing that may not help is the use of
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
613 # if..elif versus a lookup/dispatch table. There may be value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
614 # in switching that.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
615 while offset < len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
616 # Attempt to decode an item. This could be a whole value or a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
617 # special value indicating an event, such as start or end of a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
618 # collection or indefinite length type.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
619 complete, value, readcount, special = decodeitem(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
620
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
621 if readcount > 0:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
622 self.decodedbytecount += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
623
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
624 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
625 assert readcount < 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
626 return (
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
627 bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
628 offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
629 -readcount,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
630 )
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
631
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
632 offset += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
633
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
634 # No nested state. We either have a full value or beginning of a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
635 # complex value to deal with.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
636 if self._state == self._STATE_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
637 # A normal value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
638 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
639 self._decodedvalues.append(value)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
640
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
641 elif special == SPECIAL_START_ARRAY:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
642 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
643 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
644 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
645 b'v': [],
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
646 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
647 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
648 self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
649
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
650 elif special == SPECIAL_START_MAP:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
651 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
652 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
653 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
654 b'v': {},
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
655 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
656 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
657 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
658
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
659 elif special == SPECIAL_START_SET:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
660 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
661 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
662 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
663 b'v': set(),
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
664 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
665 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
666 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
667
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
668 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
669 self._state = self._STATE_WANT_BYTESTRING_CHUNK_FIRST
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
670
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
671 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
672 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
673 b'unhandled special state: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
674 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
675
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
676 # This value becomes an element of the current array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
677 elif self._state == self._STATE_WANT_ARRAY_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
678 # Simple values get appended.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
679 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
680 c = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
681 c[b'v'].append(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
682 c[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
683
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
684 # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
685
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
686 # An array nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
687 elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
688 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
689 newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
690
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
691 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
692 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
693
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
694 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
695 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
696 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
697 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
698 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
699 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
700
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
701 # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
702
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
703 # A map nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
704 elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
705 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
706 newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
707
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
708 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
709 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
710
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
711 self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
712 {b'remaining': value, b'v': newvalue}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
713 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
714
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
715 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
716
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
717 elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
718 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
719 newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
720
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
721 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
722 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
723
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
724 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
725 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
726 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
727 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
728 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
729 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
730
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
731 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
732
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
733 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
734 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
735 b'indefinite length bytestrings '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
736 b'not allowed as array values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
737 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
738
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
739 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
740 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
741 b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
742 b'expecting array value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
743 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
744
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
745 # This value becomes the key of the current map instance.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
746 elif self._state == self._STATE_WANT_MAP_KEY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
747 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
748 self._currentmapkey = value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
749 self._state = self._STATE_WANT_MAP_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
750
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
751 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
752 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
753 b'indefinite length bytestrings '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
754 b'not allowed as map keys'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
755 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
756
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
757 elif special in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
758 SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
759 SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
760 SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
761 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
762 raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
763 b'collections not supported as map keys'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
764 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
765
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
766 # We do not allow special values to be used as map keys.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
767 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
768 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
769 b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
770 b'expecting map key: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
771 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
772
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
773 # This value becomes the value of the current map key.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
774 elif self._state == self._STATE_WANT_MAP_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
775 # Simple values simply get inserted into the map.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
776 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
777 lastc = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
778 lastc[b'v'][self._currentmapkey] = value
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
779 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
780
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
781 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
782
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
783 # A new array is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
784 elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
785 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
786 newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
787
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
788 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
789 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
790
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
791 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
792 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
793 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
794 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
795 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
796 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
797
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
798 self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
799
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
800 # A new map is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
801 elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
802 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
803 newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
804
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
805 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
806 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
807
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
808 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
809 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
810 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
811 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
812 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
813 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
814
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
815 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
816
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
817 # A new set is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
818 elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
819 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
820 newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
821
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
822 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
823 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
824
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
825 self._collectionstack.append(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
826 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
827 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
828 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
829 }
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
830 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
831
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
832 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
833
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
834 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
835 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
836 b'indefinite length bytestrings not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
837 b'allowed as map values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
838 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
839
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
840 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
841 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
842 b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
843 b'expecting map value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
844 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
845
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
846 self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
847
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
848 # This value is added to the current set.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
849 elif self._state == self._STATE_WANT_SET_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
850 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
851 lastc = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
852 lastc[b'v'].add(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
853 lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
854
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
855 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
856 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
857 b'indefinite length bytestrings not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
858 b'allowed as set values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
859 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
860
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
861 elif special in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
862 SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
863 SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
864 SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
865 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
866 raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
867 b'collections not allowed as set values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
868 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
869
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
870 # We don't allow non-trivial types to exist as set values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
871 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
872 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
873 b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
874 b'expecting set value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
875 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
876
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
877 # This value represents the first chunk in an indefinite length
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
878 # bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
879 elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_FIRST:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
880 # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
881 if special == SPECIAL_NONE:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
882 self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
883 bytestringchunk(value, first=True)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
884 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
885
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
886 self._state = self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
887
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
888 # The end of stream marker. This means it is an empty
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
889 # indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
890 elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
891 # We /could/ convert this to a b''. But we want to preserve
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
892 # the nature of the underlying data so consumers expecting
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
893 # an indefinite length bytestring get one.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
894 self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
895 bytestringchunk(b'', first=True, last=True)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
896 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
897
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
898 # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
899 # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
900 assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
901 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
902
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
903 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
904 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
905 b'unexpected special value when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
906 b'expecting bytestring chunk: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
907 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
908
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
909 # This value represents the non-initial chunk in an indefinite
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
910 # length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
911 elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
912 # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
913 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
914 self._decodedvalues.append(bytestringchunk(value))
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
915
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
916 # The end of stream marker.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
917 elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
918 self._decodedvalues.append(bytestringchunk(b'', last=True))
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
919
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
920 # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
921 # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
922 assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
923 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
924
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
925 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
926 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
927 b'unexpected special value when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
928 b'expecting bytestring chunk: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
929 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
930
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
931 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
932 raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
933 b'unhandled decoder state: %d' % self._state
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
934 )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
935
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
936 # We could have just added the final value in a collection. End
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
937 # all complete collections at the top of the stack.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
938 while True:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
939 # Bail if we're not waiting on a new collection item.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
940 if self._state not in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
941 self._STATE_WANT_ARRAY_VALUE,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
942 self._STATE_WANT_MAP_KEY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
943 self._STATE_WANT_SET_VALUE,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
944 ):
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
945 break
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
946
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
947 # Or we are expecting more items for this collection.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
948 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
949
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
950 if lastc[b'remaining']:
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
951 break
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
952
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
953 # The collection at the top of the stack is complete.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
954
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
955 # Discard it, as it isn't needed for future items.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
956 self._collectionstack.pop()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
957
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
958 # If this is a nested collection, we don't emit it, since it
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
959 # will be emitted by its parent collection. But we do need to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
960 # update state to reflect what the new top-most collection
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
961 # on the stack is.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
962 if self._collectionstack:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
963 self._state = {
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
964 list: self._STATE_WANT_ARRAY_VALUE,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
965 dict: self._STATE_WANT_MAP_KEY,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
966 set: self._STATE_WANT_SET_VALUE,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
967 }[type(self._collectionstack[-1][b'v'])]
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
968
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
969 # If this is the root collection, emit it.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
970 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
971 self._decodedvalues.append(lastc[b'v'])
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
972 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
973
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
974 return (
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
975 bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
976 offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
977 0,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
978 )
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
979
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
980 def getavailable(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
981 """Returns an iterator over fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
982
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
983 Once values are retrieved, they won't be available on the next call.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
984 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
985
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
986 l = list(self._decodedvalues)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
987 self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
988 return l
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
989
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
990
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
991 class bufferingdecoder(object):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
992 """A CBOR decoder that buffers undecoded input.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
993
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
994 This is a glorified wrapper around ``sansiodecoder`` that adds a buffering
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
995 layer. All input that isn't consumed by ``sansiodecoder`` will be buffered
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
996 and concatenated with any new input that arrives later.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
997
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
998 TODO consider adding limits as to the maximum amount of data that can
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
999 be buffered.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1000 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
1001
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1002 def __init__(self):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1003 self._decoder = sansiodecoder()
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1004 self._chunks = []
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1005 self._wanted = 0
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1006
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1007 def decode(self, b):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1008 """Attempt to decode bytes to CBOR values.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1009
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1010 Returns a tuple with the following fields:
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1011
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1012 * Bool indicating whether new values are available for retrieval.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1013 * Integer number of bytes decoded from the new input.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1014 * Integer number of bytes wanted to decode the next value.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1015 """
40124
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
1016 # We /might/ be able to support passing a bytearray all the
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
1017 # way through. For now, let's cheat.
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
1018 if isinstance(b, bytearray):
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
1019 b = bytes(b)
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
1020
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1021 # Our strategy for buffering is to aggregate the incoming chunks in a
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1022 # list until we've received enough data to decode the next item.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1023 # This is slightly more complicated than using an ``io.BytesIO``
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1024 # or continuously concatenating incoming data. However, because it
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1025 # isn't constantly reallocating backing memory for a growing buffer,
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1026 # it prevents excessive memory thrashing and is significantly faster,
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1027 # especially in cases where the percentage of input chunks that don't
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1028 # decode into a full item is high.
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1029
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1030 if self._chunks:
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1031 # A previous call said we needed N bytes to decode the next item.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1032 # But this call doesn't provide enough data. We buffer the incoming
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1033 # chunk without attempting to decode.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1034 if len(b) < self._wanted:
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1035 self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1036 self._wanted -= len(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1037 return False, 0, self._wanted
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1038
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1039 # Else we may have enough data to decode the next item. Aggregate
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1040 # old data with new and reset the buffer.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1041 newlen = len(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1042 self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1043 b = b''.join(self._chunks)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1044 self._chunks = []
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1045 oldlen = len(b) - newlen
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1046
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1047 else:
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1048 oldlen = 0
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1049
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1050 available, readcount, wanted = self._decoder.decode(b)
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1051 self._wanted = wanted
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1052
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1053 if readcount < len(b):
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
1054 self._chunks.append(b[readcount:])
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1055
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1056 return available, readcount - oldlen, wanted
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1057
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1058 def getavailable(self):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1059 return self._decoder.getavailable()
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
1060
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
1061
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1062 def decodeall(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1063 """Decode all CBOR items present in an iterable of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1064
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1065 In addition to regular decode errors, raises CBORDecodeError if the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1066 entirety of the passed buffer does not fully decode to complete CBOR
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1067 values. This includes failure to decode any value, incomplete collection
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1068 types, incomplete indefinite length items, and extra data at the end of
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1069 the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1070 """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1071 if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1072 return []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1073
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1074 decoder = sansiodecoder()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1075
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1076 havevalues, readcount, wantbytes = decoder.decode(b)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1077
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1078 if readcount != len(b):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1079 raise CBORDecodeError(b'input data not fully consumed')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1080
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1081 if decoder.inprogress:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1082 raise CBORDecodeError(b'input data not complete')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1083
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
1084 return decoder.getavailable()