mercurial-scm/hg-stable: contrib/python-zstandard/tests/test_compressor

annotate contrib/python-zstandard/tests/test_compressor_fuzzing.py @ 43999:de7838053207

zstandard: vendor python-zstandard 0.13.0 Version 0.13.0 of the package was just released. It contains an upgraded zstd C library which can result in some performance wins, official support for Python 3.8, and a blackened code base. There were no meaningful code or functionality changes in this release of python-zstandard: just reformatting and an upgraded zstd library version. So the diff seems much larger than what it is. Files were added without modifications. The clang-format-ignorelist file was updated to reflect a new header file in the zstd distribution. # no-check-commit because 3rd party code has different style guidelines Differential Revision: https://phab.mercurial-scm.org/D7770

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Sat, 28 Dec 2019 09:55:45 -0800
parents	675775c33ab6
children	5e84a96d865b

rev	line source
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 import io
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 import os
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	3 import unittest
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 try:
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 import hypothesis
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7 import hypothesis.strategies as strategies
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 except ImportError:
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	9 raise unittest.SkipTest("hypothesis not available")
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	11 import zstandard as zstd
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	13 from .common import (
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 make_cffi,
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	15 NonClosingBytesIO,
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 random_input_data,
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	17 TestCase,
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	18 )
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	19
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	21 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	23 class TestCompressor_stream_reader_fuzzing(TestCase):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	24 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	25 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	26 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	27 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	28 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	29 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	30 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	31 read_size=strategies.integers(-1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	32 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	33 def test_stream_source_read(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	34 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	35 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	36
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	37 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	38 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	39
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	40 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	41 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	42 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	43 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	44 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	45 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	46 chunk = reader.read(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	47 if not chunk:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	48 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	49
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	50 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	51
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	52 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	53
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	54 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	55 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	56 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	57 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	58 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	59 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	60 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	61 read_size=strategies.integers(-1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	62 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	63 def test_buffer_source_read(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	64 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	65 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	66
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	67 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	68 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	69
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	70 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	71 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	72 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	73 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	74 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	75 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	76 chunk = reader.read(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	77 if not chunk:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	78 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	79
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	80 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	81
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	82 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	83
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	84 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	85 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	86 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	87 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	88 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	89 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	90 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	91 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	92 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	93 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	94 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	95 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	96 def test_stream_source_read_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	97 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	98 ):
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	99 refctx = zstd.ZstdCompressor(level=level)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	100 ref_frame = refctx.compress(original)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	101
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	102 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	103 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	104 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	105 ) as reader:
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	106 chunks = []
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	107 while True:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	108 read_size = read_sizes.draw(strategies.integers(-1, 16384))
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	109 chunk = reader.read(read_size)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	110 if not chunk and read_size:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	111 break
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	112
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	113 chunks.append(chunk)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	114
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	115 self.assertEqual(b"".join(chunks), ref_frame)
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	116
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	117 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	118 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	119 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	120 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	121 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	122 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	123 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	124 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	125 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	126 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	127 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	128 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	129 def test_buffer_source_read_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	130 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	131 ):
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	132
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	133 refctx = zstd.ZstdCompressor(level=level)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	134 ref_frame = refctx.compress(original)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	135
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	136 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	137 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	138 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	139 ) as reader:
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	140 chunks = []
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	141 while True:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	142 read_size = read_sizes.draw(strategies.integers(-1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	143 chunk = reader.read(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	144 if not chunk and read_size:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	145 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	146
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	147 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	148
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	149 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	150
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	151 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	152 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	153 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	154 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	155 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	156 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	157 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	158 read_size=strategies.integers(1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	159 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	160 def test_stream_source_readinto(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	161 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	162 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	163
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	164 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	165 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	166 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	167 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	168 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	169 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	170 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	171 count = reader.readinto(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	172
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	173 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	174 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	175
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	176 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	177
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	178 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	179
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	180 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	181 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	182 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	183 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	184 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	185 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	186 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	187 read_size=strategies.integers(1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	188 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	189 def test_buffer_source_readinto(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	190
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	191 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	192 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	193
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	194 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	195 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	196 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	197 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	198 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	199 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	200 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	201 count = reader.readinto(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	202
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	203 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	204 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	205
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	206 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	207
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	208 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	209
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	210 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	211 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	212 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	213 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	214 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	215 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	216 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	217 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	218 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	219 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	220 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	221 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	222 def test_stream_source_readinto_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	223 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	224 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	225 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	226 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	227
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	228 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	229 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	230 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	231 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	232 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	233 while True:
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	234 read_size = read_sizes.draw(strategies.integers(1, 16384))
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	235 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	236 count = reader.readinto(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	237
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	238 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	239 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	240
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	241 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	242
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	243 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	244
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	245 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	246 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	247 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	248 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	249 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	250 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	251 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	252 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	253 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	254 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	255 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	256 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	257 def test_buffer_source_readinto_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	258 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	259 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	260
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	261 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	262 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	263
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	264 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	265 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	266 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	267 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	268 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	269 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	270 read_size = read_sizes.draw(strategies.integers(1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	271 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	272 count = reader.readinto(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	273
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	274 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	275 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	276
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	277 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	278
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	279 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	280
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	281 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	282 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	283 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	284 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	285 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	286 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	287 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	288 read_size=strategies.integers(-1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	289 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	290 def test_stream_source_read1(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	291 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	292 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	293
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	294 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	295 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	296
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	297 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	298 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	299 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	300 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	301 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	302 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	303 chunk = reader.read1(read_size)
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	304 if not chunk:
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	305 break
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	306
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	307 chunks.append(chunk)
b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	308
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	309 self.assertEqual(b"".join(chunks), ref_frame)
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	310
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	311 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	312 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	313 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	314 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	315 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	316 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	317 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	318 read_size=strategies.integers(-1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	319 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	320 def test_buffer_source_read1(self, original, level, source_read_size, read_size):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	321 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	322 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	323
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	324 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	325 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	326
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	327 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	328 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	329 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	330 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	331 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	332 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	333 chunk = reader.read1(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	334 if not chunk:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	335 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	336
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	337 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	338
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	339 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	340
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	341 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	342 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	343 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	344 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	345 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	346 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	347 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	348 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	349 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	350 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	351 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	352 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	353 def test_stream_source_read1_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	354 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	355 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	356 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	357 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	358
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	359 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	360 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	361 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	362 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	363 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	364 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	365 read_size = read_sizes.draw(strategies.integers(-1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	366 chunk = reader.read1(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	367 if not chunk and read_size:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	368 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	369
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	370 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	371
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	372 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	373
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	374 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	375 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	376 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	377 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	378 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	379 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	380 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	381 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	382 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	383 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	384 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	385 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	386 def test_buffer_source_read1_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	387 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	388 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	389
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	390 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	391 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	392
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	393 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	394 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	395 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	396 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	397 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	398 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	399 read_size = read_sizes.draw(strategies.integers(-1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	400 chunk = reader.read1(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	401 if not chunk and read_size:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	402 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	403
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	404 chunks.append(chunk)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	405
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	406 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	407
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	408 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	409 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	410 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	411 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	412 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	413 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	414 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	415 read_size=strategies.integers(1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	416 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	417 def test_stream_source_readinto1(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	418 self, original, level, source_read_size, read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	419 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	420 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	421 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	422
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	423 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	424 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	425
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	426 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	427 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	428 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	429 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	430 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	431 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	432 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	433 count = reader.readinto1(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	434
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	435 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	436 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	437
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	438 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	439
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	440 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	441
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	442 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	443 suppress_health_check=[hypothesis.HealthCheck.large_base_example]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	444 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	445 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	446 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	447 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	448 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	449 read_size=strategies.integers(1, zstd.COMPRESSION_RECOMMENDED_OUTPUT_SIZE),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	450 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	451 def test_buffer_source_readinto1(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	452 self, original, level, source_read_size, read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	453 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	454 if read_size == 0:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	455 read_size = -1
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	456
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	457 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	458 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	459
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	460 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	461 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	462 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	463 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	464 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	465 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	466 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	467 count = reader.readinto1(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	468
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	469 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	470 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	471
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	472 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	473
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	474 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	475
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	476 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	477 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	478 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	479 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	480 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	481 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	482 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	483 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	484 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	485 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	486 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	487 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	488 def test_stream_source_readinto1_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	489 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	490 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	491 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	492 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	493
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	494 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	495 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	496 io.BytesIO(original), size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	497 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	498 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	499 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	500 read_size = read_sizes.draw(strategies.integers(1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	501 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	502 count = reader.readinto1(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	503
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	504 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	505 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	506
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	507 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	508
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	509 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	510
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	511 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	512 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	513 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	514 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	515 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	516 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	517 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	518 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	519 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	520 source_read_size=strategies.integers(1, 16384),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	521 read_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	522 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	523 def test_buffer_source_readinto1_variance(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	524 self, original, level, source_read_size, read_sizes
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	525 ):
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	526
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	527 refctx = zstd.ZstdCompressor(level=level)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	528 ref_frame = refctx.compress(original)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	529
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	530 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	531 with cctx.stream_reader(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	532 original, size=len(original), read_size=source_read_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	533 ) as reader:
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	534 chunks = []
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	535 while True:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	536 read_size = read_sizes.draw(strategies.integers(1, 16384))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	537 b = bytearray(read_size)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	538 count = reader.readinto1(b)
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	539
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	540 if not count:
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	541 break
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	542
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	543 chunks.append(bytes(b[0:count]))
675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	544
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	545 self.assertEqual(b"".join(chunks), ref_frame)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	546
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	547
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	548 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	549 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	550 class TestCompressor_stream_writer_fuzzing(TestCase):
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	551 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	552 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	553 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	554 write_size=strategies.integers(min_value=1, max_value=1048576),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	555 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	556 def test_write_size_variance(self, original, level, write_size):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	557 refctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	558 ref_frame = refctx.compress(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	559
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	560 cctx = zstd.ZstdCompressor(level=level)
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	561 b = NonClosingBytesIO()
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	562 with cctx.stream_writer(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	563 b, size=len(original), write_size=write_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	564 ) as compressor:
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	565 compressor.write(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	566
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	567 self.assertEqual(b.getvalue(), ref_frame)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	568
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	569
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	570 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	571 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	572 class TestCompressor_copy_stream_fuzzing(TestCase):
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	573 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	574 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	575 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	576 read_size=strategies.integers(min_value=1, max_value=1048576),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	577 write_size=strategies.integers(min_value=1, max_value=1048576),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	578 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	579 def test_read_write_size_variance(self, original, level, read_size, write_size):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	580 refctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	581 ref_frame = refctx.compress(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	582
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	583 cctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	584 source = io.BytesIO(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	585 dest = io.BytesIO()
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	586
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	587 cctx.copy_stream(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	588 source, dest, size=len(original), read_size=read_size, write_size=write_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	589 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	590
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	591 self.assertEqual(dest.getvalue(), ref_frame)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	592
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	593
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	594 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	595 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	596 class TestCompressor_compressobj_fuzzing(TestCase):
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	597 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	598 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	599 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	600 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	601 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	602 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	603 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	604 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	605 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	606 chunk_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	607 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	608 def test_random_input_sizes(self, original, level, chunk_sizes):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	609 refctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	610 ref_frame = refctx.compress(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	611
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	612 cctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	613 cobj = cctx.compressobj(size=len(original))
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	614
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	615 chunks = []
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	616 i = 0
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	617 while True:
37495 b1fb341d8a61 zstandard: vendor python-zstandard 0.9.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 31799 diff changeset	618 chunk_size = chunk_sizes.draw(strategies.integers(1, 4096))
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	619 source = original[i : i + chunk_size]
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	620 if not source:
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	621 break
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	622
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	623 chunks.append(cobj.compress(source))
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	624 i += chunk_size
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	625
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	626 chunks.append(cobj.flush())
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	627
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	628 self.assertEqual(b"".join(chunks), ref_frame)
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	629
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	630 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	631 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	632 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	633 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	634 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	635 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	636 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	637 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	638 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	639 chunk_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	640 flushes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	641 )
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	642 def test_flush_block(self, original, level, chunk_sizes, flushes):
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	643 cctx = zstd.ZstdCompressor(level=level)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	644 cobj = cctx.compressobj()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	645
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	646 dctx = zstd.ZstdDecompressor()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	647 dobj = dctx.decompressobj()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	648
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	649 compressed_chunks = []
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	650 decompressed_chunks = []
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	651 i = 0
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	652 while True:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	653 input_size = chunk_sizes.draw(strategies.integers(1, 4096))
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	654 source = original[i : i + input_size]
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	655 if not source:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	656 break
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	657
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	658 i += input_size
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	659
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	660 chunk = cobj.compress(source)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	661 compressed_chunks.append(chunk)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	662 decompressed_chunks.append(dobj.decompress(chunk))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	663
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	664 if not flushes.draw(strategies.booleans()):
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	665 continue
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	666
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	667 chunk = cobj.flush(zstd.COMPRESSOBJ_FLUSH_BLOCK)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	668 compressed_chunks.append(chunk)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	669 decompressed_chunks.append(dobj.decompress(chunk))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	670
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	671 self.assertEqual(b"".join(decompressed_chunks), original[0:i])
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	672
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	673 chunk = cobj.flush(zstd.COMPRESSOBJ_FLUSH_FINISH)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	674 compressed_chunks.append(chunk)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	675 decompressed_chunks.append(dobj.decompress(chunk))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	676
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	677 self.assertEqual(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	678 dctx.decompress(b"".join(compressed_chunks), max_output_size=len(original)),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	679 original,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	680 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	681 self.assertEqual(b"".join(decompressed_chunks), original)
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	682
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	683
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	684 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	685 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	686 class TestCompressor_read_to_iter_fuzzing(TestCase):
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	687 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	688 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	689 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	690 read_size=strategies.integers(min_value=1, max_value=4096),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	691 write_size=strategies.integers(min_value=1, max_value=4096),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	692 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	693 def test_read_write_size_variance(self, original, level, read_size, write_size):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	694 refcctx = zstd.ZstdCompressor(level=level)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	695 ref_frame = refcctx.compress(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	696
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	697 source = io.BytesIO(original)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	698
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	699 cctx = zstd.ZstdCompressor(level=level)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	700 chunks = list(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	701 cctx.read_to_iter(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	702 source, size=len(original), read_size=read_size, write_size=write_size
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	703 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	704 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	705
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	706 self.assertEqual(b"".join(chunks), ref_frame)
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	707
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	708
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	709 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	710 class TestCompressor_multi_compress_to_buffer_fuzzing(TestCase):
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	711 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	712 original=strategies.lists(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	713 strategies.sampled_from(random_input_data()), min_size=1, max_size=1024
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	714 ),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	715 threads=strategies.integers(min_value=1, max_value=8),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	716 use_dict=strategies.booleans(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	717 )
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	718 def test_data_equivalence(self, original, threads, use_dict):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	719 kwargs = {}
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	720
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	721 # Use a content dictionary because it is cheap to create.
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	722 if use_dict:
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	723 kwargs["dict_data"] = zstd.ZstdCompressionDict(original[0])
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	724
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	725 cctx = zstd.ZstdCompressor(level=1, write_checksum=True, **kwargs)
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	726
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	727 if not hasattr(cctx, "multi_compress_to_buffer"):
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	728 self.skipTest("multi_compress_to_buffer not available")
42070 675775c33ab6 zstandard: vendor python-zstandard 0.11 Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	729
31799 e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	730 result = cctx.multi_compress_to_buffer(original, threads=-1)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	731
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	732 self.assertEqual(len(result), len(original))
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	733
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	734 # The frame produced via the batch APIs may not be bit identical to that
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	735 # produced by compress() because compression parameters are adjusted
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	736 # from the first input in batch mode. So the only thing we can do is
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	737 # verify the decompressed data matches the input.
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	738 dctx = zstd.ZstdDecompressor(**kwargs)
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	739
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	740 for i, frame in enumerate(result):
e0dc40530c5a zstd: vendor python-zstandard 0.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	741 self.assertEqual(dctx.decompress(frame), original[i])
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	742
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	743
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	744 @unittest.skipUnless("ZSTD_SLOW_TESTS" in os.environ, "ZSTD_SLOW_TESTS not set")
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	745 @make_cffi
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	746 class TestCompressor_chunker_fuzzing(TestCase):
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	747 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	748 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	749 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	750 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	751 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	752 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	753 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	754 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	755 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	756 chunk_size=strategies.integers(min_value=1, max_value=32 * 1048576),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	757 input_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	758 )
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	759 def test_random_input_sizes(self, original, level, chunk_size, input_sizes):
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	760 cctx = zstd.ZstdCompressor(level=level)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	761 chunker = cctx.chunker(chunk_size=chunk_size)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	762
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	763 chunks = []
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	764 i = 0
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	765 while True:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	766 input_size = input_sizes.draw(strategies.integers(1, 4096))
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	767 source = original[i : i + input_size]
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	768 if not source:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	769 break
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	770
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	771 chunks.extend(chunker.compress(source))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	772 i += input_size
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	773
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	774 chunks.extend(chunker.finish())
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	775
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	776 dctx = zstd.ZstdDecompressor()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	777
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	778 self.assertEqual(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	779 dctx.decompress(b"".join(chunks), max_output_size=len(original)), original
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	780 )
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	781
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	782 self.assertTrue(all(len(chunk) == chunk_size for chunk in chunks[:-1]))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	783
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	784 @hypothesis.settings(
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	785 suppress_health_check=[
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	786 hypothesis.HealthCheck.large_base_example,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	787 hypothesis.HealthCheck.too_slow,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	788 ]
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	789 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	790 @hypothesis.given(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	791 original=strategies.sampled_from(random_input_data()),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	792 level=strategies.integers(min_value=1, max_value=5),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	793 chunk_size=strategies.integers(min_value=1, max_value=32 * 1048576),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	794 input_sizes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	795 flushes=strategies.data(),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	796 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	797 def test_flush_block(self, original, level, chunk_size, input_sizes, flushes):
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	798 cctx = zstd.ZstdCompressor(level=level)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	799 chunker = cctx.chunker(chunk_size=chunk_size)
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	800
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	801 dctx = zstd.ZstdDecompressor()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	802 dobj = dctx.decompressobj()
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	803
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	804 compressed_chunks = []
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	805 decompressed_chunks = []
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	806 i = 0
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	807 while True:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	808 input_size = input_sizes.draw(strategies.integers(1, 4096))
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	809 source = original[i : i + input_size]
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	810 if not source:
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	811 break
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	812
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	813 i += input_size
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	814
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	815 chunks = list(chunker.compress(source))
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	816 compressed_chunks.extend(chunks)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	817 decompressed_chunks.append(dobj.decompress(b"".join(chunks)))
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	818
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	819 if not flushes.draw(strategies.booleans()):
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	820 continue
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	821
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	822 chunks = list(chunker.flush())
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	823 compressed_chunks.extend(chunks)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	824 decompressed_chunks.append(dobj.decompress(b"".join(chunks)))
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	825
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	826 self.assertEqual(b"".join(decompressed_chunks), original[0:i])
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	827
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	828 chunks = list(chunker.finish())
73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	829 compressed_chunks.extend(chunks)
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	830 decompressed_chunks.append(dobj.decompress(b"".join(chunks)))
40122 73fef626dae3 zstandard: vendor python-zstandard 0.10.1 Gregory Szorc <gregory.szorc@gmail.com> parents: 37495 diff changeset	831
43999 de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	832 self.assertEqual(
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	833 dctx.decompress(b"".join(compressed_chunks), max_output_size=len(original)),
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	834 original,
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	835 )
de7838053207 zstandard: vendor python-zstandard 0.13.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 42070 diff changeset	836 self.assertEqual(b"".join(decompressed_chunks), original)

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/python-zstandard/tests/test_compressor_fuzzing.py @ 43999:de7838053207