mercurial-scm/hg-stable: mercurial/utils/cborutil.py annotate

annotate mercurial/utils/cborutil.py @ 53040:cdd7bf612c7b stable tip

bundle-spec: properly format boolean parameter (issue6960) This was breaking automatic clone bundle generation. This changeset fixes it and add a test to catch it in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 11 Mar 2025 02:29:42 +0100
parents	8a2091a2f974
children

rev	line source
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # cborutil.py - CBOR extensions
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 49037 diff changeset	8 from __future__ import annotations
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 import struct
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	11 import typing
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	13 if typing.TYPE_CHECKING:
279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	14 from typing import (
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	15 Iterable,
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	16 Iterator,
279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	17 )
39473 8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39440 diff changeset	18
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	19 # Very short very of RFC 7049...
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 # Each item begins with a byte. The 3 high bits of that byte denote the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 # "major type." The lower 5 bits denote the "subtype." Each major type
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 # has its own encoding mechanism.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 #
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 # Most types have lengths. However, bytestring, string, array, and map
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 # can be indefinite length. These are denotes by a subtype with value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 # Sub-components of those types then come afterwards and are terminated
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 # by a "break" byte.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 MAJOR_TYPE_UINT = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31 MAJOR_TYPE_NEGINT = 1
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 MAJOR_TYPE_BYTESTRING = 2
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 MAJOR_TYPE_STRING = 3
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 MAJOR_TYPE_ARRAY = 4
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 MAJOR_TYPE_MAP = 5
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 MAJOR_TYPE_SEMANTIC = 6
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 MAJOR_TYPE_SPECIAL = 7
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 SUBTYPE_MASK = 0b00011111
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	41 SUBTYPE_FALSE = 20
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	42 SUBTYPE_TRUE = 21
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	43 SUBTYPE_NULL = 22
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 SUBTYPE_HALF_FLOAT = 25
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 SUBTYPE_SINGLE_FLOAT = 26
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46 SUBTYPE_DOUBLE_FLOAT = 27
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 SUBTYPE_INDEFINITE = 31
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	49 SEMANTIC_TAG_FINITE_SET = 258
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	50
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 # Indefinite types begin with their major type ORd with information value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 BEGIN_INDEFINITE_BYTESTRING = struct.pack(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	53 '>B', MAJOR_TYPE_BYTESTRING << 5 \| SUBTYPE_INDEFINITE
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	54 )
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55 BEGIN_INDEFINITE_ARRAY = struct.pack(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	56 '>B', MAJOR_TYPE_ARRAY << 5 \| SUBTYPE_INDEFINITE
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	57 )
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 BEGIN_INDEFINITE_MAP = struct.pack(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	59 '>B', MAJOR_TYPE_MAP << 5 \| SUBTYPE_INDEFINITE
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	60 )
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	62 ENCODED_LENGTH_1 = struct.Struct('>B')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	63 ENCODED_LENGTH_2 = struct.Struct('>BB')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	64 ENCODED_LENGTH_3 = struct.Struct('>BH')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	65 ENCODED_LENGTH_4 = struct.Struct('>BL')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	66 ENCODED_LENGTH_5 = struct.Struct('>BQ')
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	68 # The break ends an indefinite length item.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	69 BREAK = b'\xff'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70 BREAK_INT = 255
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	71
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	72
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	73 def encodelength(majortype: int, length: int) -> bytes:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	74 """Obtain a value encoding the major type and its length."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 if length < 24:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 return ENCODED_LENGTH_1.pack(majortype << 5 \| length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 elif length < 256:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 return ENCODED_LENGTH_2.pack(majortype << 5 \| 24, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 elif length < 65536:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 return ENCODED_LENGTH_3.pack(majortype << 5 \| 25, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 elif length < 4294967296:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 return ENCODED_LENGTH_4.pack(majortype << 5 \| 26, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 return ENCODED_LENGTH_5.pack(majortype << 5 \| 27, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	86
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	87 def streamencodebytestring(v: bytes) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	88 yield encodelength(MAJOR_TYPE_BYTESTRING, len(v))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	89 yield v
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	90
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	91
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	92 def streamencodebytestringfromiter(it: Iterable[bytes]) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93 """Convert an iterator of chunks to an indefinite bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	94
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	95 Given an input that is iterable and each element in the iterator is
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	96 representable as bytes, emit an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	97 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98 yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100 for chunk in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	104 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	105
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	106
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	107 def streamencodeindefinitebytestring(
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	108 source, chunksize: int = 65536
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	109 ) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110 """Given a large source buffer, emit as an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	111
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	112 This is a generator of chunks constituting the encoded CBOR data.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	113 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	114 yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	115
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	116 i = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	117 l = len(source)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	118
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	119 while True:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	120 chunk = source[i : i + chunksize]
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	121 i += len(chunk)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	122
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123 yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	124 yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	125
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126 if i >= l:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 break
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	128
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	131
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	132 def streamencodeint(v: int) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133 if v >= 18446744073709551616 or v < -18446744073709551616:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	134 raise ValueError('big integers not supported')
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136 if v >= 0:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 yield encodelength(MAJOR_TYPE_UINT, v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 yield encodelength(MAJOR_TYPE_NEGINT, abs(v) - 1)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	141
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	142 def streamencodearray(l) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 """Encode a known size iterable to an array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145 yield encodelength(MAJOR_TYPE_ARRAY, len(l))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147 for i in l:
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	148 yield from streamencode(i)
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	149
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	150
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	151 def streamencodearrayfromiter(it) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	152 """Encode an iterator of items to an indefinite length array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	153
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 yield BEGIN_INDEFINITE_ARRAY
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 for i in it:
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	157 yield from streamencode(i)
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	158
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	159 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	161
37942 2ae6a3134362 cborutil: port to Python 3 Augie Fackler <augie@google.com> parents: 37711 diff changeset	162 def _mixedtypesortkey(v):
2ae6a3134362 cborutil: port to Python 3 Augie Fackler <augie@google.com> parents: 37711 diff changeset	163 return type(v).__name__, v
2ae6a3134362 cborutil: port to Python 3 Augie Fackler <augie@google.com> parents: 37711 diff changeset	164
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	165
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	166 def streamencodeset(s) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 # https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml defines
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 # semantic tag 258 for finite sets.
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	169 yield encodelength(MAJOR_TYPE_SEMANTIC, SEMANTIC_TAG_FINITE_SET)
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	171 yield from streamencodearray(sorted(s, key=_mixedtypesortkey))
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	173
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	174 def streamencodemap(d: dict) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 """Encode dictionary to a generator.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 Does not supporting indefinite length dictionaries.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 yield encodelength(MAJOR_TYPE_MAP, len(d))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	181 for key, value in sorted(d.items(), key=lambda x: _mixedtypesortkey(x[0])):
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	182 yield from streamencode(key)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	183 yield from streamencode(value)
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	185
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	186 def streamencodemapfromiter(it: Iterable) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 """Given an iterable of (key, value), encode to an indefinite length map."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 yield BEGIN_INDEFINITE_MAP
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 for key, value in it:
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	191 yield from streamencode(key)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52659 diff changeset	192 yield from streamencode(value)
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	196
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	197 def streamencodebool(b: bool) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	198 # major type 7, simple value 20 and 21.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199 yield b'\xf5' if b else b'\xf4'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	201
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	202 def streamencodenone(v: None) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	203 # major type 7, simple value 22.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	204 yield b'\xf6'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	205
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	206
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207 STREAM_ENCODERS = {
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	208 bytes: streamencodebytestring,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209 int: streamencodeint,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210 list: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 tuple: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212 dict: streamencodemap,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 set: streamencodeset,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 bool: streamencodebool,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215 type(None): streamencodenone,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 }
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	218
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	219 def streamencode(v) -> Iterator[bytes]:
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 """Encode a value in a streaming manner.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	221
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 Given an input object, encode it to CBOR recursively.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224 Returns a generator of CBOR encoded bytes. There is no guarantee
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 that each emitted chunk fully decodes to a value or sub-value.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 Encoding is deterministic - unordered collections are sorted.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228 """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229 fn = STREAM_ENCODERS.get(v.__class__)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231 if not fn:
42176 b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	232 # handle subtypes such as encoding.localstr and util.sortdict
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	233 for ty in STREAM_ENCODERS:
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	234 if not isinstance(v, ty):
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	235 continue
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	236 fn = STREAM_ENCODERS[ty]
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	237 break
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	238
b6387a65851d cborutil: fix streamencode() to handle subtypes Yuya Nishihara <yuya@tcha.org> parents: 40125 diff changeset	239 if not fn:
52659 fb8932685031 cborutil: fix an error interpolating str into bytes Matt Harbison <matt_harbison@yahoo.com> parents: 51915 diff changeset	240 raise ValueError('do not know how to encode %s' % type(v))
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242 return fn(v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	243
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	244
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	245 class CBORDecodeError(Exception):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	246 """Represents an error decoding CBOR."""
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	247
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	248
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	249 def _elementtointeger(b, i: int) -> int:
49033 bce8f66d3045 cborutil: remove Python 2 definition of _elementtointeger() Gregory Szorc <gregory.szorc@gmail.com> parents: 49023 diff changeset	250 return b[i]
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	251
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	252
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	253 STRUCT_BIG_UBYTE = struct.Struct('>B')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	254 STRUCT_BIG_USHORT = struct.Struct(b'>H')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	255 STRUCT_BIG_ULONG = struct.Struct(b'>L')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	256 STRUCT_BIG_ULONGLONG = struct.Struct(b'>Q')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	257
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	258 SPECIAL_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	259 SPECIAL_START_INDEFINITE_BYTESTRING = 1
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	260 SPECIAL_START_ARRAY = 2
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	261 SPECIAL_START_MAP = 3
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	262 SPECIAL_START_SET = 4
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	263 SPECIAL_INDEFINITE_BREAK = 5
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	264
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	265
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	266 def decodeitem(b, offset: int = 0):
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	267 """Decode a new CBOR value from a buffer at offset.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	268
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	269 This function attempts to decode up to one complete CBOR value
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	270 from ``b`` starting at offset ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	271
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	272 The beginning of a collection (such as an array, map, set, or
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	273 indefinite length bytestring) counts as a single value. For these
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	274 special cases, a state flag will indicate that a special value was seen.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	275
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	276 When called, the function either returns a decoded value or gives
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	277 a hint as to how many more bytes are needed to do so. By calling
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	278 the function repeatedly given a stream of bytes, the caller can
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	279 build up the original values.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	280
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	281 Returns a tuple with the following elements:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	282
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	283 * Bool indicating whether a complete value was decoded.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	284 * A decoded value if first value is True otherwise None
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	285 * Integer number of bytes. If positive, the number of bytes
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	286 read. If negative, the number of bytes we need to read to
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	287 decode this value or the next chunk in this value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	288 * One of the ``SPECIAL_*`` constants indicating special treatment
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	289 for this value. ``SPECIAL_NONE`` means this is a fully decoded
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	290 simple value (such as an integer or bool).
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	291 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	292
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	293 initial = _elementtointeger(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	294 offset += 1
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	295
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	296 majortype = initial >> 5
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	297 subtype = initial & SUBTYPE_MASK
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	298
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	299 if majortype == MAJOR_TYPE_UINT:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	300 complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	301
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	302 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	303 return True, value, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	304 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	305 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	306
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	307 elif majortype == MAJOR_TYPE_NEGINT:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	308 # Negative integers are the same as UINT except inverted minus 1.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	309 complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	310
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	311 if complete:
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	312 assert value is not None # help pytype
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	313 return True, -value - 1, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	314 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	315 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	316
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	317 elif majortype == MAJOR_TYPE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	318 # Beginning of bytestrings are treated as uints in order to
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	319 # decode their length, which may be indefinite.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	320 complete, size, readcount = decodeuint(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	321 subtype, b, offset, allowindefinite=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	322 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	323
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	324 # We don't know the size of the bytestring. It must be a definitive
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	325 # length since the indefinite subtype would be encoded in the initial
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	326 # byte.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	327 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	328 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	329
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	330 # We know the length of the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	331 if size is not None:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	332 # And the data is available in the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	333 if offset + readcount + size <= len(b):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	334 value = b[offset + readcount : offset + readcount + size]
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	335 return True, value, readcount + size + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	336
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	337 # And we need more data in order to return the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	338 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	339 wanted = len(b) - offset - readcount - size
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	340 return False, None, wanted, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	341
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	342 # It is an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	343 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	344 return True, None, 1, SPECIAL_START_INDEFINITE_BYTESTRING
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	345
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	346 elif majortype == MAJOR_TYPE_STRING:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	347 raise CBORDecodeError('string major type not supported')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	348
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	349 elif majortype == MAJOR_TYPE_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	350 # Beginning of arrays are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	351 # length. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	352 complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	353
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	354 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	355 return True, size, readcount + 1, SPECIAL_START_ARRAY
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	356 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	357 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	358
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	359 elif majortype == MAJOR_TYPE_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	360 # Beginning of maps are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	361 # number of elements. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	362 complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	363
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	364 if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	365 return True, size, readcount + 1, SPECIAL_START_MAP
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	366 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	367 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	368
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	369 elif majortype == MAJOR_TYPE_SEMANTIC:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	370 # Semantic tag value is read the same as a uint.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	371 complete, tagvalue, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	372
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	373 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	374 return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	375
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	376 # This behavior here is a little wonky. The main type being "decorated"
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	377 # by this semantic tag follows. A more robust parser would probably emit
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	378 # a special flag indicating this as a semantic tag and let the caller
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	379 # deal with the types that follow. But since we don't support many
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	380 # semantic tags, it is easier to deal with the special cases here and
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	381 # hide complexity from the caller. If we add support for more semantic
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	382 # tags, we should probably move semantic tag handling into the caller.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	383 if tagvalue == SEMANTIC_TAG_FINITE_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	384 if offset + readcount >= len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	385 return False, None, -1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	386
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	387 complete, size, readcount2, special = decodeitem(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	388 b, offset + readcount
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	389 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	390
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	391 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	392 return False, None, readcount2, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	393
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	394 if special != SPECIAL_START_ARRAY:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	395 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	396 'expected array after finite set semantic tag'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	397 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	398
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	399 return True, size, readcount + readcount2 + 1, SPECIAL_START_SET
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	400
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	401 else:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	402 raise CBORDecodeError('semantic tag %d not allowed' % tagvalue)
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	403
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	404 elif majortype == MAJOR_TYPE_SPECIAL:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	405 # Only specific values for the information field are allowed.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	406 if subtype == SUBTYPE_FALSE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	407 return True, False, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	408 elif subtype == SUBTYPE_TRUE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	409 return True, True, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	410 elif subtype == SUBTYPE_NULL:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	411 return True, None, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	412 elif subtype == SUBTYPE_INDEFINITE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	413 return True, None, 1, SPECIAL_INDEFINITE_BREAK
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	414 # If value is 24, subtype is in next byte.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	415 else:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	416 raise CBORDecodeError('special type %d not allowed' % subtype)
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	417 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	418 assert False
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	419
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	420
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	421 def decodeuint(
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	422 subtype: int, b: bytes, offset: int = 0, allowindefinite: bool = False
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	423 ):
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	424 """Decode an unsigned integer.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	425
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	426 ``subtype`` is the lower 5 bits from the initial byte CBOR item
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	427 "header." ``b`` is a buffer containing bytes. ``offset`` points to
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	428 the index of the first byte after the byte that ``subtype`` was
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	429 derived from.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	430
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	431 ``allowindefinite`` allows the special indefinite length value
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	432 indicator.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	433
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	434 Returns a 3-tuple of (successful, value, count).
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	435
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	436 The first element is a bool indicating if decoding completed. The 2nd
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	437 is the decoded integer value or None if not fully decoded or the subtype
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	438 is 31 and ``allowindefinite`` is True. The 3rd value is the count of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	439 If positive, it is the number of additional bytes decoded. If negative,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	440 it is the number of additional bytes needed to decode this value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	441 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	442
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	443 # Small values are inline.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	444 if subtype < 24:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	445 return True, subtype, 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	446 # Indefinite length specifier.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	447 elif subtype == 31:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	448 if allowindefinite:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	449 return True, None, 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	450 else:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	451 raise CBORDecodeError('indefinite length uint not allowed here')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	452 elif subtype >= 28:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	453 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	454 'unsupported subtype on integer type: %d' % subtype
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	455 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	456
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	457 if subtype == 24:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	458 s = STRUCT_BIG_UBYTE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	459 elif subtype == 25:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	460 s = STRUCT_BIG_USHORT
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	461 elif subtype == 26:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	462 s = STRUCT_BIG_ULONG
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	463 elif subtype == 27:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	464 s = STRUCT_BIG_ULONGLONG
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	465 else:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	466 raise CBORDecodeError('bounds condition checking violation')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	467
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	468 if len(b) - offset >= s.size:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	469 return True, s.unpack_from(b, offset)[0], s.size
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	470 else:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	471 return False, None, len(b) - offset - s.size
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	472
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	473
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	474 class bytestringchunk(bytes):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	475 """Represents a chunk/segment in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	476
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	477 This behaves like a ``bytes`` but in addition has the ``isfirst``
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	478 and ``islast`` attributes indicating whether this chunk is the first
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	479 or last in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	480 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	481
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	482 isfirst: bool
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	483 islast: bool
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	484
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	485 def __new__(cls, v, first: bool = False, last: bool = False):
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	486 self = bytes.__new__(cls, v)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	487 self.isfirst = first
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	488 self.islast = last
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	489
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	490 return self
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	491
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	492
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49033 diff changeset	493 class sansiodecoder:
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	494 """A CBOR decoder that doesn't perform its own I/O.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	495
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	496 To use, construct an instance and feed it segments containing
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	497 CBOR-encoded bytes via ``decode()``. The return value from ``decode()``
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	498 indicates whether a fully-decoded value is available, how many bytes
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	499 were consumed, and offers a hint as to how many bytes should be fed
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	500 in next time to decode the next value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	501
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	502 The decoder assumes it will decode N discrete CBOR values, not just
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	503 a single value. i.e. if the bytestream contains uints packed one after
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	504 the other, the decoder will decode them all, rather than just the initial
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	505 one.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	506
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	507 When ``decode()`` indicates a value is available, call ``getavailable()``
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	508 to return all fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	509
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	510 ``decode()`` can partially decode input. It is up to the caller to keep
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	511 track of what data was consumed and to pass unconsumed data in on the
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	512 next invocation.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	513
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	514 The decoder decodes atomically at the item level. See ``decodeitem()``.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	515 If an item cannot be fully decoded, the decoder won't record it as
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	516 partially consumed. Instead, the caller will be instructed to pass in
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	517 the initial bytes of this item on the next invocation. This does result
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	518 in some redundant parsing. But the overhead should be minimal.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	519
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	520 This decoder only supports a subset of CBOR as required by Mercurial.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	521 It lacks support for:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	522
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	523 * Indefinite length arrays
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	524 * Indefinite length maps
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	525 * Use of indefinite length bytestrings as keys or values within
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	526 arrays, maps, or sets.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	527 * Nested arrays, maps, or sets within sets
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	528 * Any semantic tag that isn't a mathematical finite set
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	529 * Floating point numbers
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	530 * Undefined special value
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	531
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	532 CBOR types are decoded to Python types as follows:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	533
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	534 uint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	535 negint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	536 bytestring -> bytes
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	537 map -> dict
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	538 array -> list
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	539 True -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	540 False -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	541 null -> None
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	542 indefinite length bytestring chunk -> [bytestringchunk]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	543
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	544 The only non-obvious mapping here is an indefinite length bytestring
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	545 to the ``bytestringchunk`` type. This is to facilitate streaming
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	546 indefinite length bytestrings out of the decoder and to differentiate
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	547 a regular bytestring from an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	548 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	549
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	550 _STATE_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	551 _STATE_WANT_MAP_KEY = 1
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	552 _STATE_WANT_MAP_VALUE = 2
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	553 _STATE_WANT_ARRAY_VALUE = 3
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	554 _STATE_WANT_SET_VALUE = 4
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	555 _STATE_WANT_BYTESTRING_CHUNK_FIRST = 5
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	556 _STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT = 6
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	557
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	558 def __init__(self) -> None:
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	559 # TODO add support for limiting size of bytestrings
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	560 # TODO add support for limiting number of keys / values in collections
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	561 # TODO add support for limiting size of buffered partial values
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	562
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	563 self.decodedbytecount = 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	564
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	565 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	566
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	567 # Stack of active nested collections. Each entry is a dict describing
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	568 # the collection.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	569 self._collectionstack = []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	570
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	571 # Fully decoded key to use for the current map.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	572 self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	573
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	574 # Fully decoded values available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	575 self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	576
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	577 @property
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	578 def inprogress(self) -> bool:
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	579 """Whether the decoder has partially decoded a value."""
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	580 return self._state != self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	581
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	582 def decode(self, b, offset: int = 0) -> tuple[bool, int, int]:
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	583 """Attempt to decode bytes from an input buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	584
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	585 ``b`` is a collection of bytes and ``offset`` is the byte
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	586 offset within that buffer from which to begin reading data.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	587
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	588 ``b`` must support ``len()`` and accessing bytes slices via
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	589 ``__slice__``. Typically ``bytes`` instances are used.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	590
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	591 Returns a tuple with the following fields:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	592
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	593 * Bool indicating whether values are available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	594 * Integer indicating the number of bytes that were fully consumed,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	595 starting from ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	596 * Integer indicating the number of bytes that are desired for the
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	597 next call in order to decode an item.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	598 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	599 if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	600 return bool(self._decodedvalues), 0, 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	601
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	602 initialoffset = offset
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	603
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	604 # We could easily split the body of this loop into a function. But
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	605 # Python performance is sensitive to function calls and collections
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	606 # are composed of many items. So leaving as a while loop could help
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	607 # with performance. One thing that may not help is the use of
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	608 # if..elif versus a lookup/dispatch table. There may be value
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	609 # in switching that.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	610 while offset < len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	611 # Attempt to decode an item. This could be a whole value or a
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	612 # special value indicating an event, such as start or end of a
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	613 # collection or indefinite length type.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	614 complete, value, readcount, special = decodeitem(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	615
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	616 if readcount > 0:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	617 self.decodedbytecount += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	618
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	619 if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	620 assert readcount < 0
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	621 return (
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	622 bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	623 offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	624 -readcount,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	625 )
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	626
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	627 offset += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	628
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	629 # No nested state. We either have a full value or beginning of a
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	630 # complex value to deal with.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	631 if self._state == self._STATE_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	632 # A normal value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	633 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	634 self._decodedvalues.append(value)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	635
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	636 elif special == SPECIAL_START_ARRAY:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	637 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	638 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	639 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	640 b'v': [],
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	641 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	642 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	643 self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	644
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	645 elif special == SPECIAL_START_MAP:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	646 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	647 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	648 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	649 b'v': {},
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	650 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	651 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	652 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	653
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	654 elif special == SPECIAL_START_SET:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	655 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	656 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	657 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	658 b'v': set(),
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	659 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	660 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	661 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	662
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	663 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	664 self._state = self._STATE_WANT_BYTESTRING_CHUNK_FIRST
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	665
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	666 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	667 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	668 'unhandled special state: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	669 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	670
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	671 # This value becomes an element of the current array.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	672 elif self._state == self._STATE_WANT_ARRAY_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	673 # Simple values get appended.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	674 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	675 c = self._collectionstack[-1]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	676 c[b'v'].append(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	677 c[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	678
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	679 # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	680
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	681 # An array nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	682 elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	683 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	684 newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	685
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	686 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	687 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	688
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	689 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	690 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	691 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	692 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	693 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	694 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	695
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	696 # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	697
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	698 # A map nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	699 elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	700 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	701 newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	702
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	703 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	704 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	705
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	706 self._collectionstack.append(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	707 {b'remaining': value, b'v': newvalue}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	708 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	709
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	710 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	711
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	712 elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	713 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	714 newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	715
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	716 lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	717 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	718
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	719 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	720 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	721 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	722 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	723 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	724 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	725
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	726 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	727
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	728 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	729 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	730 'indefinite length bytestrings '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	731 'not allowed as array values'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	732 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	733
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	734 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	735 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	736 'unhandled special item when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	737 'expecting array value: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	738 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	739
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	740 # This value becomes the key of the current map instance.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	741 elif self._state == self._STATE_WANT_MAP_KEY:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	742 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	743 self._currentmapkey = value
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	744 self._state = self._STATE_WANT_MAP_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	745
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	746 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	747 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	748 'indefinite length bytestrings '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	749 'not allowed as map keys'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	750 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	751
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	752 elif special in (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	753 SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	754 SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	755 SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	756 ):
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	757 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	758 'collections not supported as map keys'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	759 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	760
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	761 # We do not allow special values to be used as map keys.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	762 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	763 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	764 'unhandled special item when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	765 'expecting map key: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	766 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	767
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	768 # This value becomes the value of the current map key.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	769 elif self._state == self._STATE_WANT_MAP_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	770 # Simple values simply get inserted into the map.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	771 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	772 lastc = self._collectionstack[-1]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	773 lastc[b'v'][self._currentmapkey] = value
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	774 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	775
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	776 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	777
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	778 # A new array is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	779 elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	780 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	781 newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	782
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	783 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	784 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	785
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	786 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	787 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	788 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	789 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	790 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	791 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	792
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	793 self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	794
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	795 # A new map is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	796 elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	797 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	798 newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	799
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	800 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	801 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	802
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	803 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	804 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	805 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	806 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	807 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	808 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	809
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	810 self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	811
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	812 # A new set is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	813 elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	814 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	815 newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	816
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	817 lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	818 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	819
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	820 self._collectionstack.append(
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	821 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	822 b'remaining': value,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	823 b'v': newvalue,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 43554 diff changeset	824 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	825 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	826
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	827 self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	828
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	829 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	830 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	831 'indefinite length bytestrings not '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	832 'allowed as map values'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	833 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	834
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	835 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	836 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	837 'unhandled special item when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	838 'expecting map value: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	839 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	840
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	841 self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	842
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	843 # This value is added to the current set.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	844 elif self._state == self._STATE_WANT_SET_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	845 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	846 lastc = self._collectionstack[-1]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	847 lastc[b'v'].add(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	848 lastc[b'remaining'] -= 1
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	849
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	850 elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	851 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	852 'indefinite length bytestrings not '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	853 'allowed as set values'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	854 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	855
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	856 elif special in (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	857 SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	858 SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	859 SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	860 ):
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	861 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	862 'collections not allowed as set values'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	863 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	864
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	865 # We don't allow non-trivial types to exist as set values.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	866 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	867 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	868 'unhandled special item when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	869 'expecting set value: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	870 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	871
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	872 # This value represents the first chunk in an indefinite length
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	873 # bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	874 elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_FIRST:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	875 # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	876 if special == SPECIAL_NONE:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	877 self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	878 bytestringchunk(value, first=True)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	879 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	880
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	881 self._state = self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	882
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	883 # The end of stream marker. This means it is an empty
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	884 # indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	885 elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	886 # We /could/ convert this to a b''. But we want to preserve
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	887 # the nature of the underlying data so consumers expecting
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	888 # an indefinite length bytestring get one.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	889 self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	890 bytestringchunk(b'', first=True, last=True)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	891 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	892
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	893 # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	894 # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	895 assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	896 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	897
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	898 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	899 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	900 'unexpected special value when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	901 'expecting bytestring chunk: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	902 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	903
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	904 # This value represents the non-initial chunk in an indefinite
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	905 # length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	906 elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	907 # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	908 if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	909 self._decodedvalues.append(bytestringchunk(value))
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	910
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	911 # The end of stream marker.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	912 elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	913 self._decodedvalues.append(bytestringchunk(b'', last=True))
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	914
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	915 # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	916 # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	917 assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	918 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	919
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	920 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	921 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	922 'unexpected special value when '
8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	923 'expecting bytestring chunk: %d' % special
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	924 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	925
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	926 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	927 raise CBORDecodeError(
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	928 'unhandled decoder state: %d' % self._state
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	929 )
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	930
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	931 # We could have just added the final value in a collection. End
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	932 # all complete collections at the top of the stack.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	933 while True:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	934 # Bail if we're not waiting on a new collection item.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	935 if self._state not in (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	936 self._STATE_WANT_ARRAY_VALUE,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	937 self._STATE_WANT_MAP_KEY,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	938 self._STATE_WANT_SET_VALUE,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	939 ):
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	940 break
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	941
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	942 # Or we are expecting more items for this collection.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	943 lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	944
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	945 if lastc[b'remaining']:
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	946 break
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	947
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	948 # The collection at the top of the stack is complete.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	949
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	950 # Discard it, as it isn't needed for future items.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	951 self._collectionstack.pop()
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	952
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	953 # If this is a nested collection, we don't emit it, since it
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	954 # will be emitted by its parent collection. But we do need to
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	955 # update state to reflect what the new top-most collection
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	956 # on the stack is.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	957 if self._collectionstack:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	958 self._state = {
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	959 list: self._STATE_WANT_ARRAY_VALUE,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	960 dict: self._STATE_WANT_MAP_KEY,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	961 set: self._STATE_WANT_SET_VALUE,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	962 }[type(self._collectionstack[-1][b'v'])]
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	963
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	964 # If this is the root collection, emit it.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	965 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	966 self._decodedvalues.append(lastc[b'v'])
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	967 self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	968
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	969 return (
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	970 bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	971 offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	972 0,
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	973 )
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	974
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	975 def getavailable(self):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	976 """Returns an iterator over fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	977
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	978 Once values are retrieved, they won't be available on the next call.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	979 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	980
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	981 l = list(self._decodedvalues)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	982 self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	983 return l
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	984
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	985
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49033 diff changeset	986 class bufferingdecoder:
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	987 """A CBOR decoder that buffers undecoded input.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	988
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	989 This is a glorified wrapper around ``sansiodecoder`` that adds a buffering
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	990 layer. All input that isn't consumed by ``sansiodecoder`` will be buffered
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	991 and concatenated with any new input that arrives later.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	992
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	993 TODO consider adding limits as to the maximum amount of data that can
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	994 be buffered.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	995 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	996
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	997 _decoder: sansiodecoder
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	998 _chunks: list
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	999 _wanted: int
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	1000
5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	1001 def __init__(self) -> None:
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1002 self._decoder = sansiodecoder()
40031 62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1003 self._chunks = []
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1004 self._wanted = 0
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1005
52711 5e09c6b5b795 typing: add type annotations to most of `mercurial/utils/cborutil.py` Matt Harbison <matt_harbison@yahoo.com> parents: 52709 diff changeset	1006 def decode(self, b) -> tuple[bool, int, int]:
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1007 """Attempt to decode bytes to CBOR values.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1008
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1009 Returns a tuple with the following fields:
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1010
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1011 * Bool indicating whether new values are available for retrieval.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1012 * Integer number of bytes decoded from the new input.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1013 * Integer number of bytes wanted to decode the next value.
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1014 """
40125 b638219a23c3 cborutil: cast bytearray to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 40031 diff changeset	1015 # We /might/ be able to support passing a bytearray all the
b638219a23c3 cborutil: cast bytearray to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 40031 diff changeset	1016 # way through. For now, let's cheat.
b638219a23c3 cborutil: cast bytearray to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 40031 diff changeset	1017 if isinstance(b, bytearray):
b638219a23c3 cborutil: cast bytearray to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 40031 diff changeset	1018 b = bytes(b)
b638219a23c3 cborutil: cast bytearray to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 40031 diff changeset	1019
40031 62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1020 # Our strategy for buffering is to aggregate the incoming chunks in a
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1021 # list until we've received enough data to decode the next item.
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1022 # This is slightly more complicated than using an ``io.BytesIO``
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1023 # or continuously concatenating incoming data. However, because it
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1024 # isn't constantly reallocating backing memory for a growing buffer,
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1025 # it prevents excessive memory thrashing and is significantly faster,
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1026 # especially in cases where the percentage of input chunks that don't
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1027 # decode into a full item is high.
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1028
40031 62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1029 if self._chunks:
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1030 # A previous call said we needed N bytes to decode the next item.
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1031 # But this call doesn't provide enough data. We buffer the incoming
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1032 # chunk without attempting to decode.
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1033 if len(b) < self._wanted:
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1034 self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1035 self._wanted -= len(b)
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1036 return False, 0, self._wanted
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1037
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1038 # Else we may have enough data to decode the next item. Aggregate
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1039 # old data with new and reset the buffer.
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1040 newlen = len(b)
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1041 self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1042 b = b''.join(self._chunks)
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1043 self._chunks = []
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1044 oldlen = len(b) - newlen
62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1045
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1046 else:
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1047 oldlen = 0
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1048
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1049 available, readcount, wanted = self._decoder.decode(b)
40031 62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1050 self._wanted = wanted
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1051
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1052 if readcount < len(b):
40031 62160d3077cd cborutil: change buffering strategy Gregory Szorc <gregory.szorc@gmail.com> parents: 39473 diff changeset	1053 self._chunks.append(b[readcount:])
39440 babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1054
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1055 return available, readcount - oldlen, wanted
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1056
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1057 def getavailable(self):
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1058 return self._decoder.getavailable()
babad5ebaf0a cborutil: add a buffering decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 39439 diff changeset	1059
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42176 diff changeset	1060
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1061 def decodeall(b):
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1062 """Decode all CBOR items present in an iterable of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1063
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1064 In addition to regular decode errors, raises CBORDecodeError if the
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1065 entirety of the passed buffer does not fully decode to complete CBOR
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1066 values. This includes failure to decode any value, incomplete collection
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1067 types, incomplete indefinite length items, and extra data at the end of
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1068 the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1069 """
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1070 if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1071 return []
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1072
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1073 decoder = sansiodecoder()
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1074
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1075 havevalues, readcount, wantbytes = decoder.decode(b)
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1076
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1077 if readcount != len(b):
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	1078 raise CBORDecodeError('input data not fully consumed')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1079
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1080 if decoder.inprogress:
52712 8a2091a2f974 cborutil: unbyteify string args to builtin Error classes Matt Harbison <matt_harbison@yahoo.com> parents: 52711 diff changeset	1081 raise CBORDecodeError('input data not complete')
39438 aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1082
aeb551a3bb8a cborutil: implement sans I/O decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 37942 diff changeset	1083 return decoder.getavailable()

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/utils/cborutil.py @ 53040:cdd7bf612c7b stable tip