Mercurial > public > mercurial-scm > hg-stable
comparison mercurial/utils/compression.py @ 46789:471cd86c8eb4
bundle: optional multithreaded compression, ATM zstd-only
Compression type can be a huge chunk of "hg bundle", especially when
using the higher compression levels. With level=22 and threads=7, the
NetBSD test repository took 28:39 wall time and 157:47 user time.
Before, level=22 would take 129:20 wall time and 129:07 user time.
Differential Revision: https://phab.mercurial-scm.org/D9283
| author | Joerg Sonnenberger <joerg@bec.de> |
|---|---|
| date | Sun, 08 Nov 2020 20:17:09 +0100 |
| parents | 84130fd74a22 |
| children | 6000f5b25c9b |
comparison
equal
deleted
inserted
replaced
| 46788:c94fa884240b | 46789:471cd86c8eb4 |
|---|---|
| 683 opts = opts or {} | 683 opts = opts or {} |
| 684 # zstd level 3 is almost always significantly faster than zlib | 684 # zstd level 3 is almost always significantly faster than zlib |
| 685 # while providing no worse compression. It strikes a good balance | 685 # while providing no worse compression. It strikes a good balance |
| 686 # between speed and compression. | 686 # between speed and compression. |
| 687 level = opts.get(b'level', 3) | 687 level = opts.get(b'level', 3) |
| 688 # default to single-threaded compression | |
| 689 threads = opts.get(b'threads', 0) | |
| 688 | 690 |
| 689 zstd = self._module | 691 zstd = self._module |
| 690 z = zstd.ZstdCompressor(level=level).compressobj() | 692 z = zstd.ZstdCompressor(level=level, threads=threads).compressobj() |
| 691 for chunk in it: | 693 for chunk in it: |
| 692 data = z.compress(chunk) | 694 data = z.compress(chunk) |
| 693 if data: | 695 if data: |
| 694 yield data | 696 yield data |
| 695 | 697 |