mercurial-scm/hg: contrib/python-zstandard/c-ext/decompressor.c annotate

annotate contrib/python-zstandard/c-ext/decompressor.c @ 30895:c32454d69b85

zstd: vendor python-zstandard 0.7.0 Commit 3054ae3a66112970a091d3939fee32c2d0c1a23e from https://github.com/indygreg/python-zstandard is imported without modifications (other than removing unwanted files). The vendored zstd library within has been upgraded from 1.1.2 to 1.1.3. This version introduced new APIs for threads, thread pools, multi-threaded compression, and a new dictionary builder (COVER). These features are not yet used by python-zstandard (or Mercurial for that matter). However, that will likely change in the next python-zstandard release (and I think there are opportunities for Mercurial to take advantage of the multi-threaded APIs). Relevant to Mercurial, the CFFI bindings are now fully implemented. This means zstd should "just work" with PyPy (although I haven't tried). The python-zstandard test suite also runs all tests against both the C extension and CFFI bindings to ensure feature parity. There is also a "decompress_content_dict_chain()" API. This was derived from discussions with Yann Collet on list about alternate ways of encoding delta chains. The change most relevant to Mercurial is a performance enhancement in the simple decompression API to reuse a data structure across operations. This makes decompression of multiple inputs significantly faster. (This scenario occurs when reading revlog delta chains, for example.) Using python-zstandard's bench.py to measure the performance difference... On changelog chunks in the mozilla-unified repo: decompress discrete decompress() reuse zctx 1.262243 wall; 1.260000 CPU; 1.260000 user; 0.000000 sys 170.43 MB/s (best of 3) 0.949106 wall; 0.950000 CPU; 0.950000 user; 0.000000 sys 226.66 MB/s (best of 4) decompress discrete dict decompress() reuse zctx 0.692170 wall; 0.690000 CPU; 0.690000 user; 0.000000 sys 310.80 MB/s (best of 5) 0.437088 wall; 0.440000 CPU; 0.440000 user; 0.000000 sys 492.17 MB/s (best of 7) On manifest chunks in the mozilla-unified repo: decompress discrete decompress() reuse zctx 1.367284 wall; 1.370000 CPU; 1.370000 user; 0.000000 sys 274.01 MB/s (best of 3) 1.086831 wall; 1.080000 CPU; 1.080000 user; 0.000000 sys 344.72 MB/s (best of 3) decompress discrete dict decompress() reuse zctx 0.993272 wall; 0.990000 CPU; 0.990000 user; 0.000000 sys 377.19 MB/s (best of 3) 0.678651 wall; 0.680000 CPU; 0.680000 user; 0.000000 sys 552.06 MB/s (best of 5) That should make reads on zstd revlogs a bit faster ;) # no-check-commit

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Tue, 07 Feb 2017 23:24:47 -0800
parents	08fa3a76a080
children	e0dc40530c5a

rev	line source
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 /**
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 * Copyright (c) 2016-present, Gregory Szorc
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 * All rights reserved.
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 *
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 * This software may be modified and distributed under the terms
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 * of the BSD license. See the LICENSE file for details.
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7 */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 #include "python-zstandard.h"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 extern PyObject* ZstdError;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 ZSTD_DStream* DStream_from_ZstdDecompressor(ZstdDecompressor* decompressor) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 ZSTD_DStream* dstream;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 void* dictData = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 size_t dictSize = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17 size_t zresult;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	18
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	19 dstream = ZSTD_createDStream();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20 if (!dstream) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 PyErr_SetString(ZstdError, "could not create DStream");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 if (decompressor->dict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 dictData = decompressor->dict->dictData;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 dictSize = decompressor->dict->dictSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 if (dictData) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31 zresult = ZSTD_initDStream_usingDict(dstream, dictData, dictSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 else {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 zresult = ZSTD_initDStream(dstream);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 if (ZSTD_isError(zresult)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 PyErr_Format(ZstdError, "could not initialize DStream: %s",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 ZSTD_getErrorName(zresult));
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43 return dstream;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46 PyDoc_STRVAR(Decompressor__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 "ZstdDecompressor(dict_data=None)\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49 "Create an object used to perform Zstandard decompression.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 "An instance can perform multiple decompression operations."
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 static int Decompressor_init(ZstdDecompressor* self, PyObject* args, PyObject* kwargs) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55 static char* kwlist[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 "dict_data",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 NULL
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 ZstdCompressionDict* dict = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	62 self->dctx = NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 self->dict = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 self->ddict = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	66 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "\|O!:ZstdDecompressor", kwlist,
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67 &ZstdCompressionDictType, &dict)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	68 return -1;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	69 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	71 /* TODO lazily initialize the reference ZSTD_DCtx on first use since
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 not instances of ZstdDecompressor will use a ZSTD_DCtx. */
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	73 self->dctx = ZSTD_createDCtx();
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	74 if (!self->dctx) {
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 PyErr_NoMemory();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 if (dict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 self->dict = dict;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 Py_INCREF(dict);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 return 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	86 except:
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	87 if (self->dctx) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	88 ZSTD_freeDCtx(self->dctx);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	89 self->dctx = NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	90 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	91
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	92 return -1;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	94
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	95 static void Decompressor_dealloc(ZstdDecompressor* self) {
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	96 if (self->dctx) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	97 ZSTD_freeDCtx(self->dctx);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100 Py_XDECREF(self->dict);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102 if (self->ddict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103 ZSTD_freeDDict(self->ddict);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	104 self->ddict = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	105 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	106
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	107 PyObject_Del(self);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	108 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	109
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110 PyDoc_STRVAR(Decompressor_copy_stream__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	111 "copy_stream(ifh, ofh[, read_size=default, write_size=default]) -- decompress data between streams\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	112 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	113 "Compressed data will be read from ``ifh``, decompressed, and written to\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	114 "``ofh``. ``ifh`` must have a ``read(size)`` method. ``ofh`` must have a\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	115 "``write(data)`` method.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	116 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	117 "The optional ``read_size`` and ``write_size`` arguments control the chunk\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	118 "size of data that is ``read()`` and ``write()`` between streams. They default\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	119 "to the default input and output sizes of zstd decompressor streams.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	120 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	121
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	122 static PyObject* Decompressor_copy_stream(ZstdDecompressor* self, PyObject* args, PyObject* kwargs) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123 static char* kwlist[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	124 "ifh",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	125 "ofh",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126 "read_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 "write_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	128 NULL
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	131 PyObject* source;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132 PyObject* dest;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133 size_t inSize = ZSTD_DStreamInSize();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 size_t outSize = ZSTD_DStreamOutSize();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135 ZSTD_DStream* dstream;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136 ZSTD_inBuffer input;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 ZSTD_outBuffer output;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 Py_ssize_t totalRead = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 Py_ssize_t totalWrite = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 char* readBuffer;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141 Py_ssize_t readSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 PyObject* readResult;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 PyObject* res = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144 size_t zresult = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145 PyObject* writeResult;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 PyObject* totalReadPy;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147 PyObject* totalWritePy;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	148
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	149 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO\|kk:copy_stream", kwlist,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	150 &source, &dest, &inSize, &outSize)) {
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	151 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	152 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	153
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 if (!PyObject_HasAttrString(source, "read")) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155 PyErr_SetString(PyExc_ValueError, "first argument must have a read() method");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	158
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	159 if (!PyObject_HasAttrString(dest, "write")) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160 PyErr_SetString(PyExc_ValueError, "second argument must have a write() method");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	161 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163
30830 08fa3a76a080 zstd: prevent potential free() of uninitialized memory Gregory Szorc <gregory.szorc@gmail.com> parents: 30435 diff changeset	164 /* Prevent free on uninitialized memory in finally. */
08fa3a76a080 zstd: prevent potential free() of uninitialized memory Gregory Szorc <gregory.szorc@gmail.com> parents: 30435 diff changeset	165 output.dst = NULL;
08fa3a76a080 zstd: prevent potential free() of uninitialized memory Gregory Szorc <gregory.szorc@gmail.com> parents: 30435 diff changeset	166
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 dstream = DStream_from_ZstdDecompressor(self);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 if (!dstream) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	169 res = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170 goto finally;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 output.dst = PyMem_Malloc(outSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 if (!output.dst) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 PyErr_NoMemory();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176 res = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 goto finally;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 output.size = outSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180 output.pos = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182 /* Read source stream until EOF */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	183 while (1) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 readResult = PyObject_CallMethod(source, "read", "n", inSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 if (!readResult) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	186 PyErr_SetString(ZstdError, "could not read() from source");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 goto finally;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 PyBytes_AsStringAndSize(readResult, &readBuffer, &readSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 /* If no data was read, we're at EOF. */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193 if (0 == readSize) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 break;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	197 totalRead += readSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	198
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199 /* Send data to decompressor */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200 input.src = readBuffer;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201 input.size = readSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202 input.pos = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	203
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	204 while (input.pos < input.size) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	205 Py_BEGIN_ALLOW_THREADS
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	206 zresult = ZSTD_decompressStream(dstream, &output, &input);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207 Py_END_ALLOW_THREADS
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	208
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209 if (ZSTD_isError(zresult)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210 PyErr_Format(ZstdError, "zstd decompressor error: %s",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 ZSTD_getErrorName(zresult));
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212 res = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 goto finally;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 if (output.pos) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217 #if PY_MAJOR_VERSION >= 3
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 writeResult = PyObject_CallMethod(dest, "write", "y#",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 #else
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 writeResult = PyObject_CallMethod(dest, "write", "s#",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	221 #endif
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 output.dst, output.pos);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224 Py_XDECREF(writeResult);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 totalWrite += output.pos;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226 output.pos = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231 /* Source stream is exhausted. Finish up. */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233 ZSTD_freeDStream(dstream);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 dstream = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 totalReadPy = PyLong_FromSsize_t(totalRead);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237 totalWritePy = PyLong_FromSsize_t(totalWrite);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238 res = PyTuple_Pack(2, totalReadPy, totalWritePy);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239 Py_DecRef(totalReadPy);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	240 Py_DecRef(totalWritePy);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	242 finally:
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	243 if (output.dst) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	244 PyMem_Free(output.dst);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	245 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247 if (dstream) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 ZSTD_freeDStream(dstream);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 return res;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254 PyDoc_STRVAR(Decompressor_decompress__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255 "decompress(data[, max_output_size=None]) -- Decompress data in its entirety\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 "This method will decompress the entirety of the argument and return the\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 "result.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260 "The input bytes are expected to contain a full Zstandard frame (something\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 "compressed with ``ZstdCompressor.compress()`` or similar). If the input does\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 "not contain a full frame, an exception will be raised.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264 "If the frame header of the compressed data does not contain the content size\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	265 "``max_output_size`` must be specified or ``ZstdError`` will be raised. An\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	266 "allocation of size ``max_output_size`` will be performed and an attempt will\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	267 "be made to perform decompression into that buffer. If the buffer is too\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	268 "small or cannot be allocated, ``ZstdError`` will be raised. The buffer will\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	269 "be resized if it is too large.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	270 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	271 "Uncompressed data could be much larger than compressed data. As a result,\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272 "calling this function could result in a very large memory allocation being\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273 "performed to hold the uncompressed data. Therefore it is highly\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 "recommended to use a streaming decompression method instead of this one.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	277 PyObject* Decompressor_decompress(ZstdDecompressor* self, PyObject* args, PyObject* kwargs) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	278 static char* kwlist[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	279 "data",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	280 "max_output_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	281 NULL
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	282 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	283
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	284 const char* source;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	285 Py_ssize_t sourceSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	286 Py_ssize_t maxOutputSize = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	287 unsigned long long decompressedSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288 size_t destCapacity;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	289 PyObject* result = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	290 void* dictData = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291 size_t dictSize = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292 size_t zresult;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294 #if PY_MAJOR_VERSION >= 3
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	295 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y#\|n:decompress",
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	296 #else
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	297 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s#\|n:decompress",
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	298 #endif
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	299 kwlist, &source, &sourceSize, &maxOutputSize)) {
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	302
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	303 if (self->dict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304 dictData = self->dict->dictData;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 dictSize = self->dict->dictSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 if (dictData && !self->ddict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309 Py_BEGIN_ALLOW_THREADS
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	310 self->ddict = ZSTD_createDDict_byReference(dictData, dictSize);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 Py_END_ALLOW_THREADS
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313 if (!self->ddict) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314 PyErr_SetString(ZstdError, "could not create decompression dict");
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	315 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	316 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	317 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	319 decompressedSize = ZSTD_getDecompressedSize(source, sourceSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	320 /* 0 returned if content size not in the zstd frame header */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	321 if (0 == decompressedSize) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	322 if (0 == maxOutputSize) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323 PyErr_SetString(ZstdError, "input data invalid or missing content size "
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	324 "in frame header");
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	325 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	326 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327 else {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328 result = PyBytes_FromStringAndSize(NULL, maxOutputSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329 destCapacity = maxOutputSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	330 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	332 else {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 result = PyBytes_FromStringAndSize(NULL, decompressedSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334 destCapacity = decompressedSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 if (!result) {
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	338 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341 Py_BEGIN_ALLOW_THREADS
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	342 if (self->ddict) {
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	343 zresult = ZSTD_decompress_usingDDict(self->dctx,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	344 PyBytes_AsString(result), destCapacity,
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345 source, sourceSize, self->ddict);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347 else {
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	348 zresult = ZSTD_decompressDCtx(self->dctx,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	349 PyBytes_AsString(result), destCapacity, source, sourceSize);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351 Py_END_ALLOW_THREADS
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 if (ZSTD_isError(zresult)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 PyErr_Format(ZstdError, "decompression error: %s", ZSTD_getErrorName(zresult));
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	355 Py_DecRef(result);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	356 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 else if (decompressedSize && zresult != decompressedSize) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	359 PyErr_Format(ZstdError, "decompression error: decompressed %zu bytes; expected %llu",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	360 zresult, decompressedSize);
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	361 Py_DecRef(result);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	362 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	363 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364 else if (zresult < destCapacity) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 if (_PyBytes_Resize(&result, zresult)) {
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	366 Py_DecRef(result);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	367 return NULL;
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	368 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	369 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	370
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	371 return result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	372 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	373
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	374 PyDoc_STRVAR(Decompressor_decompressobj__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	375 "decompressobj()\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	376 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	377 "Incrementally feed data into a decompressor.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	378 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	379 "The returned object exposes a ``decompress(data)`` method. This makes it\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	380 "compatible with ``zlib.decompressobj`` and ``bz2.BZ2Decompressor`` so that\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	381 "callers can swap in the zstd decompressor while using the same API.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	382 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	383
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	384 static ZstdDecompressionObj* Decompressor_decompressobj(ZstdDecompressor* self) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	385 ZstdDecompressionObj* result = PyObject_New(ZstdDecompressionObj, &ZstdDecompressionObjType);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	386 if (!result) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	387 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	388 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	389
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	390 result->dstream = DStream_from_ZstdDecompressor(self);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	391 if (!result->dstream) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	392 Py_DecRef((PyObject*)result);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	393 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	394 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	395
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	396 result->decompressor = self;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	397 Py_INCREF(result->decompressor);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	398
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	399 result->finished = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	400
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	401 return result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	402 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	403
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	404 PyDoc_STRVAR(Decompressor_read_from__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	405 "read_from(reader[, read_size=default, write_size=default, skip_bytes=0])\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	406 "Read compressed data and return an iterator\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	407 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	408 "Returns an iterator of decompressed data chunks produced from reading from\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	409 "the ``reader``.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	410 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	411 "Compressed data will be obtained from ``reader`` by calling the\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	412 "``read(size)`` method of it. The source data will be streamed into a\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	413 "decompressor. As decompressed data is available, it will be exposed to the\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	414 "returned iterator.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	415 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	416 "Data is ``read()`` in chunks of size ``read_size`` and exposed to the\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	417 "iterator in chunks of size ``write_size``. The default values are the input\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	418 "and output sizes for a zstd streaming decompressor.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	419 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	420 "There is also support for skipping the first ``skip_bytes`` of data from\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	421 "the source.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	422 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	423
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	424 static ZstdDecompressorIterator* Decompressor_read_from(ZstdDecompressor* self, PyObject* args, PyObject* kwargs) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	425 static char* kwlist[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	426 "reader",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	427 "read_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	428 "write_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	429 "skip_bytes",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	430 NULL
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	431 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	432
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	433 PyObject* reader;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	434 size_t inSize = ZSTD_DStreamInSize();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	435 size_t outSize = ZSTD_DStreamOutSize();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	436 ZstdDecompressorIterator* result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	437 size_t skipBytes = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	438
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	439 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O\|kkk:read_from", kwlist,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	440 &reader, &inSize, &outSize, &skipBytes)) {
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	441 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	442 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	443
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	444 if (skipBytes >= inSize) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	445 PyErr_SetString(PyExc_ValueError,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	446 "skip_bytes must be smaller than read_size");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	447 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	448 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	449
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	450 result = PyObject_New(ZstdDecompressorIterator, &ZstdDecompressorIteratorType);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	451 if (!result) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	452 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	453 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	454
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	455 result->decompressor = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	456 result->reader = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	457 result->buffer = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	458 result->dstream = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	459 result->input.src = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	460 result->output.dst = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	461
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	462 if (PyObject_HasAttrString(reader, "read")) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	463 result->reader = reader;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	464 Py_INCREF(result->reader);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	465 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	466 else if (1 == PyObject_CheckBuffer(reader)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	467 /* Object claims it is a buffer. Try to get a handle to it. */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	468 result->buffer = PyMem_Malloc(sizeof(Py_buffer));
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	469 if (!result->buffer) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	470 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	471 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	472
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	473 memset(result->buffer, 0, sizeof(Py_buffer));
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	474
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	475 if (0 != PyObject_GetBuffer(reader, result->buffer, PyBUF_CONTIG_RO)) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	476 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	477 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	478
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	479 result->bufferOffset = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	480 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	481 else {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	482 PyErr_SetString(PyExc_ValueError,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	483 "must pass an object with a read() method or conforms to buffer protocol");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	484 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	485 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	486
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	487 result->decompressor = self;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	488 Py_INCREF(result->decompressor);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	489
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	490 result->inSize = inSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	491 result->outSize = outSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	492 result->skipBytes = skipBytes;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	493
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	494 result->dstream = DStream_from_ZstdDecompressor(self);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	495 if (!result->dstream) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	496 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	497 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	498
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	499 result->input.src = PyMem_Malloc(inSize);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	500 if (!result->input.src) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	501 PyErr_NoMemory();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	502 goto except;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	503 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	504 result->input.size = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	505 result->input.pos = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	506
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	507 result->output.dst = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	508 result->output.size = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	509 result->output.pos = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	510
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	511 result->readCount = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	512 result->finishedInput = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	513 result->finishedOutput = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	514
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	515 goto finally;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	516
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	517 except:
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	518 Py_CLEAR(result->reader);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	519
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	520 if (result->buffer) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	521 PyBuffer_Release(result->buffer);
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	522 Py_CLEAR(result->buffer);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	523 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	524
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	525 Py_CLEAR(result);
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	526
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	527 finally:
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	528
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	529 return result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	530 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	531
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	532 PyDoc_STRVAR(Decompressor_write_to__doc__,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	533 "Create a context manager to write decompressed data to an object.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	534 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	535 "The passed object must have a ``write()`` method.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	536 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	537 "The caller feeds intput data to the object by calling ``write(data)``.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	538 "Decompressed data is written to the argument given as it is decompressed.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	539 "\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	540 "An optional ``write_size`` argument defines the size of chunks to\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	541 "``write()`` to the writer. It defaults to the default output size for a zstd\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	542 "streaming decompressor.\n"
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	543 );
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	544
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	545 static ZstdDecompressionWriter* Decompressor_write_to(ZstdDecompressor* self, PyObject* args, PyObject* kwargs) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	546 static char* kwlist[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	547 "writer",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	548 "write_size",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	549 NULL
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	550 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	551
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	552 PyObject* writer;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	553 size_t outSize = ZSTD_DStreamOutSize();
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	554 ZstdDecompressionWriter* result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	555
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	556 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O\|k:write_to", kwlist,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	557 &writer, &outSize)) {
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	558 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	559 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	560
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	561 if (!PyObject_HasAttrString(writer, "write")) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	562 PyErr_SetString(PyExc_ValueError, "must pass an object with a write() method");
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	563 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	564 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	565
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	566 result = PyObject_New(ZstdDecompressionWriter, &ZstdDecompressionWriterType);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	567 if (!result) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	568 return NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	569 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	570
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	571 result->decompressor = self;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	572 Py_INCREF(result->decompressor);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	573
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	574 result->writer = writer;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	575 Py_INCREF(result->writer);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	576
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	577 result->outSize = outSize;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	578
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	579 result->entered = 0;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	580 result->dstream = NULL;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	581
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	582 return result;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	583 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	584
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	585 PyDoc_STRVAR(Decompressor_decompress_content_dict_chain__doc__,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	586 "Decompress a series of chunks using the content dictionary chaining technique\n"
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	587 );
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	588
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	589 static PyObject* Decompressor_decompress_content_dict_chain(PyObject* self, PyObject* args, PyObject* kwargs) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	590 static char* kwlist[] = {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	591 "frames",
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	592 NULL
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	593 };
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	594
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	595 PyObject* chunks;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	596 Py_ssize_t chunksLen;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	597 Py_ssize_t chunkIndex;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	598 char parity = 0;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	599 PyObject* chunk;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	600 char* chunkData;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	601 Py_ssize_t chunkSize;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	602 ZSTD_DCtx* dctx = NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	603 size_t zresult;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	604 ZSTD_frameParams frameParams;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	605 void* buffer1 = NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	606 size_t buffer1Size = 0;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	607 size_t buffer1ContentSize = 0;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	608 void* buffer2 = NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	609 size_t buffer2Size = 0;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	610 size_t buffer2ContentSize = 0;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	611 void* destBuffer = NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	612 PyObject* result = NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	613
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	614 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!:decompress_content_dict_chain",
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	615 kwlist, &PyList_Type, &chunks)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	616 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	617 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	618
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	619 chunksLen = PyList_Size(chunks);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	620 if (!chunksLen) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	621 PyErr_SetString(PyExc_ValueError, "empty input chain");
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	622 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	623 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	624
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	625 /* The first chunk should not be using a dictionary. We handle it specially. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	626 chunk = PyList_GetItem(chunks, 0);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	627 if (!PyBytes_Check(chunk)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	628 PyErr_SetString(PyExc_ValueError, "chunk 0 must be bytes");
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	629 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	630 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	631
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	632 /* We require that all chunks be zstd frames and that they have content size set. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	633 PyBytes_AsStringAndSize(chunk, &chunkData, &chunkSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	634 zresult = ZSTD_getFrameParams(&frameParams, (void*)chunkData, chunkSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	635 if (ZSTD_isError(zresult)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	636 PyErr_SetString(PyExc_ValueError, "chunk 0 is not a valid zstd frame");
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	637 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	638 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	639 else if (zresult) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	640 PyErr_SetString(PyExc_ValueError, "chunk 0 is too small to contain a zstd frame");
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	641 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	642 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	643
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	644 if (0 == frameParams.frameContentSize) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	645 PyErr_SetString(PyExc_ValueError, "chunk 0 missing content size in frame");
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	646 return NULL;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	647 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	648
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	649 dctx = ZSTD_createDCtx();
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	650 if (!dctx) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	651 PyErr_NoMemory();
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	652 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	653 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	654
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	655 buffer1Size = frameParams.frameContentSize;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	656 buffer1 = PyMem_Malloc(buffer1Size);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	657 if (!buffer1) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	658 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	659 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	660
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	661 Py_BEGIN_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	662 zresult = ZSTD_decompressDCtx(dctx, buffer1, buffer1Size, chunkData, chunkSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	663 Py_END_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	664 if (ZSTD_isError(zresult)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	665 PyErr_Format(ZstdError, "could not decompress chunk 0: %s", ZSTD_getErrorName(zresult));
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	666 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	667 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	668
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	669 buffer1ContentSize = zresult;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	670
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	671 /* Special case of a simple chain. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	672 if (1 == chunksLen) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	673 result = PyBytes_FromStringAndSize(buffer1, buffer1Size);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	674 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	675 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	676
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	677 /* This should ideally look at next chunk. But this is slightly simpler. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	678 buffer2Size = frameParams.frameContentSize;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	679 buffer2 = PyMem_Malloc(buffer2Size);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	680 if (!buffer2) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	681 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	682 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	683
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	684 /* For each subsequent chunk, use the previous fulltext as a content dictionary.
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	685 Our strategy is to have 2 buffers. One holds the previous fulltext (to be
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	686 used as a content dictionary) and the other holds the new fulltext. The
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	687 buffers grow when needed but never decrease in size. This limits the
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	688 memory allocator overhead.
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	689 */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	690 for (chunkIndex = 1; chunkIndex < chunksLen; chunkIndex++) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	691 chunk = PyList_GetItem(chunks, chunkIndex);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	692 if (!PyBytes_Check(chunk)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	693 PyErr_Format(PyExc_ValueError, "chunk %zd must be bytes", chunkIndex);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	694 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	695 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	696
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	697 PyBytes_AsStringAndSize(chunk, &chunkData, &chunkSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	698 zresult = ZSTD_getFrameParams(&frameParams, (void*)chunkData, chunkSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	699 if (ZSTD_isError(zresult)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	700 PyErr_Format(PyExc_ValueError, "chunk %zd is not a valid zstd frame", chunkIndex);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	701 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	702 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	703 else if (zresult) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	704 PyErr_Format(PyExc_ValueError, "chunk %zd is too small to contain a zstd frame", chunkIndex);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	705 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	706 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	707
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	708 if (0 == frameParams.frameContentSize) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	709 PyErr_Format(PyExc_ValueError, "chunk %zd missing content size in frame", chunkIndex);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	710 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	711 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	712
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	713 parity = chunkIndex % 2;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	714
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	715 /* This could definitely be abstracted to reduce code duplication. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	716 if (parity) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	717 /* Resize destination buffer to hold larger content. */
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	718 if (buffer2Size < frameParams.frameContentSize) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	719 buffer2Size = frameParams.frameContentSize;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	720 destBuffer = PyMem_Realloc(buffer2, buffer2Size);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	721 if (!destBuffer) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	722 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	723 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	724 buffer2 = destBuffer;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	725 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	726
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	727 Py_BEGIN_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	728 zresult = ZSTD_decompress_usingDict(dctx, buffer2, buffer2Size,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	729 chunkData, chunkSize, buffer1, buffer1ContentSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	730 Py_END_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	731 if (ZSTD_isError(zresult)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	732 PyErr_Format(ZstdError, "could not decompress chunk %zd: %s",
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	733 chunkIndex, ZSTD_getErrorName(zresult));
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	734 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	735 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	736 buffer2ContentSize = zresult;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	737 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	738 else {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	739 if (buffer1Size < frameParams.frameContentSize) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	740 buffer1Size = frameParams.frameContentSize;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	741 destBuffer = PyMem_Realloc(buffer1, buffer1Size);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	742 if (!destBuffer) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	743 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	744 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	745 buffer1 = destBuffer;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	746 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	747
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	748 Py_BEGIN_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	749 zresult = ZSTD_decompress_usingDict(dctx, buffer1, buffer1Size,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	750 chunkData, chunkSize, buffer2, buffer2ContentSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	751 Py_END_ALLOW_THREADS
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	752 if (ZSTD_isError(zresult)) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	753 PyErr_Format(ZstdError, "could not decompress chunk %zd: %s",
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	754 chunkIndex, ZSTD_getErrorName(zresult));
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	755 goto finally;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	756 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	757 buffer1ContentSize = zresult;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	758 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	759 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	760
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	761 result = PyBytes_FromStringAndSize(parity ? buffer2 : buffer1,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	762 parity ? buffer2ContentSize : buffer1ContentSize);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	763
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	764 finally:
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	765 if (buffer2) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	766 PyMem_Free(buffer2);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	767 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	768 if (buffer1) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	769 PyMem_Free(buffer1);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	770 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	771
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	772 if (dctx) {
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	773 ZSTD_freeDCtx(dctx);
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	774 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	775
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	776 return result;
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	777 }
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	778
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	779 static PyMethodDef Decompressor_methods[] = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	780 { "copy_stream", (PyCFunction)Decompressor_copy_stream, METH_VARARGS \| METH_KEYWORDS,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	781 Decompressor_copy_stream__doc__ },
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	782 { "decompress", (PyCFunction)Decompressor_decompress, METH_VARARGS \| METH_KEYWORDS,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	783 Decompressor_decompress__doc__ },
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	784 { "decompressobj", (PyCFunction)Decompressor_decompressobj, METH_NOARGS,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	785 Decompressor_decompressobj__doc__ },
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	786 { "read_from", (PyCFunction)Decompressor_read_from, METH_VARARGS \| METH_KEYWORDS,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	787 Decompressor_read_from__doc__ },
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	788 { "write_to", (PyCFunction)Decompressor_write_to, METH_VARARGS \| METH_KEYWORDS,
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	789 Decompressor_write_to__doc__ },
30895 c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	790 { "decompress_content_dict_chain", (PyCFunction)Decompressor_decompress_content_dict_chain,
c32454d69b85 zstd: vendor python-zstandard 0.7.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 30830 diff changeset	791 METH_VARARGS \| METH_KEYWORDS, Decompressor_decompress_content_dict_chain__doc__ },
30435 b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	792 { NULL, NULL }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	793 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	794
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	795 PyTypeObject ZstdDecompressorType = {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	796 PyVarObject_HEAD_INIT(NULL, 0)
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	797 "zstd.ZstdDecompressor", /* tp_name */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	798 sizeof(ZstdDecompressor), /* tp_basicsize */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	799 0, /* tp_itemsize */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	800 (destructor)Decompressor_dealloc, /* tp_dealloc */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	801 0, /* tp_print */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	802 0, /* tp_getattr */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	803 0, /* tp_setattr */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	804 0, /* tp_compare */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	805 0, /* tp_repr */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	806 0, /* tp_as_number */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	807 0, /* tp_as_sequence */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	808 0, /* tp_as_mapping */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	809 0, /* tp_hash */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	810 0, /* tp_call */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	811 0, /* tp_str */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	812 0, /* tp_getattro */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	813 0, /* tp_setattro */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	814 0, /* tp_as_buffer */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	815 Py_TPFLAGS_DEFAULT \| Py_TPFLAGS_BASETYPE, /* tp_flags */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	816 Decompressor__doc__, /* tp_doc */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	817 0, /* tp_traverse */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	818 0, /* tp_clear */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	819 0, /* tp_richcompare */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	820 0, /* tp_weaklistoffset */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	821 0, /* tp_iter */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	822 0, /* tp_iternext */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	823 Decompressor_methods, /* tp_methods */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	824 0, /* tp_members */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	825 0, /* tp_getset */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	826 0, /* tp_base */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	827 0, /* tp_dict */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	828 0, /* tp_descr_get */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	829 0, /* tp_descr_set */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	830 0, /* tp_dictoffset */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	831 (initproc)Decompressor_init, /* tp_init */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	832 0, /* tp_alloc */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	833 PyType_GenericNew, /* tp_new */
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	834 };
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	835
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	836 void decompressor_module_init(PyObject* mod) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	837 Py_TYPE(&ZstdDecompressorType) = &PyType_Type;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	838 if (PyType_Ready(&ZstdDecompressorType) < 0) {
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	839 return;
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	840 }
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	841
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	842 Py_INCREF((PyObject*)&ZstdDecompressorType);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	843 PyModule_AddObject(mod, "ZstdDecompressor",
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	844 (PyObject*)&ZstdDecompressorType);
b86a448a2965 zstd: vendor python-zstandard 0.5.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	845 }

Mercurial > public > mercurial-scm > hg

annotate contrib/python-zstandard/c-ext/decompressor.c @ 30895:c32454d69b85