Mercurial > public > mercurial-scm > hg
comparison mercurial/utils/cborutil.py @ 39413:babad5ebaf0a
cborutil: add a buffering decoder
The sansiodecoder leaves it up to the callers to feed in data that
wasn't fully consumed last time.
This commit implements a decoder that performs buffering of
leftover chunks from the previous invocation. It otherwise
behaves identically to sansiodecoder.
Differential Revision: https://phab.mercurial-scm.org/D4434
| author | Gregory Szorc <gregory.szorc@gmail.com> |
|---|---|
| date | Wed, 29 Aug 2018 14:29:01 -0700 |
| parents | a40d3da89b7d |
| children | 8d858fbf2759 |
comparison
equal
deleted
inserted
replaced
| 39412:a40d3da89b7d | 39413:babad5ebaf0a |
|---|---|
| 896 | 896 |
| 897 l = list(self._decodedvalues) | 897 l = list(self._decodedvalues) |
| 898 self._decodedvalues = [] | 898 self._decodedvalues = [] |
| 899 return l | 899 return l |
| 900 | 900 |
| 901 class bufferingdecoder(object): | |
| 902 """A CBOR decoder that buffers undecoded input. | |
| 903 | |
| 904 This is a glorified wrapper around ``sansiodecoder`` that adds a buffering | |
| 905 layer. All input that isn't consumed by ``sansiodecoder`` will be buffered | |
| 906 and concatenated with any new input that arrives later. | |
| 907 | |
| 908 TODO consider adding limits as to the maximum amount of data that can | |
| 909 be buffered. | |
| 910 """ | |
| 911 def __init__(self): | |
| 912 self._decoder = sansiodecoder() | |
| 913 self._leftover = None | |
| 914 | |
| 915 def decode(self, b): | |
| 916 """Attempt to decode bytes to CBOR values. | |
| 917 | |
| 918 Returns a tuple with the following fields: | |
| 919 | |
| 920 * Bool indicating whether new values are available for retrieval. | |
| 921 * Integer number of bytes decoded from the new input. | |
| 922 * Integer number of bytes wanted to decode the next value. | |
| 923 """ | |
| 924 | |
| 925 if self._leftover: | |
| 926 oldlen = len(self._leftover) | |
| 927 b = self._leftover + b | |
| 928 self._leftover = None | |
| 929 else: | |
| 930 b = b | |
| 931 oldlen = 0 | |
| 932 | |
| 933 available, readcount, wanted = self._decoder.decode(b) | |
| 934 | |
| 935 if readcount < len(b): | |
| 936 self._leftover = b[readcount:] | |
| 937 | |
| 938 return available, readcount - oldlen, wanted | |
| 939 | |
| 940 def getavailable(self): | |
| 941 return self._decoder.getavailable() | |
| 942 | |
| 901 def decodeall(b): | 943 def decodeall(b): |
| 902 """Decode all CBOR items present in an iterable of bytes. | 944 """Decode all CBOR items present in an iterable of bytes. |
| 903 | 945 |
| 904 In addition to regular decode errors, raises CBORDecodeError if the | 946 In addition to regular decode errors, raises CBORDecodeError if the |
| 905 entirety of the passed buffer does not fully decode to complete CBOR | 947 entirety of the passed buffer does not fully decode to complete CBOR |