comparison mercurial/revlog.py @ 38717:aa21a9ad46ea

sparse-revlog: new requirement enabled with format.sparse-revlog The meaning of the new 'sparse-revlog' requirement is that the revlogs are allowed to contain wider delta chains with larger holes between the interesting chunks. These sparse delta chains should be read in several chunks to avoid a potential explosion of memory usage. Former version won't know how to read a delta chain in several chunks. They would keep reading them in a single read, and therefore would be subject to the potential memory explosion. Hence this new requirement: only versions having support of sparse-revlog reading should be allowed to read such a revlog. Implementation of this new algorithm and tools to enable or disable the requirement will follow in the next changesets.
author Paul Morelle <paul.morelle@octobus.net>
date Mon, 04 Jun 2018 22:23:18 +0200
parents c67093e81a3e
children f8762ea73e0d
comparison
equal deleted inserted replaced
38716:c67093e81a3e 38717:aa21a9ad46ea
893 self._nodecache = {nullid: nullrev} 893 self._nodecache = {nullid: nullrev}
894 self._nodepos = None 894 self._nodepos = None
895 self._compengine = 'zlib' 895 self._compengine = 'zlib'
896 self._maxdeltachainspan = -1 896 self._maxdeltachainspan = -1
897 self._withsparseread = False 897 self._withsparseread = False
898 self._sparserevlog = False
898 self._srdensitythreshold = 0.50 899 self._srdensitythreshold = 0.50
899 self._srmingapsize = 262144 900 self._srmingapsize = 262144
900 901
901 mmapindexthreshold = None 902 mmapindexthreshold = None
902 v = REVLOG_DEFAULT_VERSION 903 v = REVLOG_DEFAULT_VERSION
921 self._compengine = opts['compengine'] 922 self._compengine = opts['compengine']
922 if 'maxdeltachainspan' in opts: 923 if 'maxdeltachainspan' in opts:
923 self._maxdeltachainspan = opts['maxdeltachainspan'] 924 self._maxdeltachainspan = opts['maxdeltachainspan']
924 if mmaplargeindex and 'mmapindexthreshold' in opts: 925 if mmaplargeindex and 'mmapindexthreshold' in opts:
925 mmapindexthreshold = opts['mmapindexthreshold'] 926 mmapindexthreshold = opts['mmapindexthreshold']
926 self._withsparseread = bool(opts.get('with-sparse-read', False)) 927 self._sparserevlog = bool(opts.get('sparse-revlog', False))
928 withsparseread = bool(opts.get('with-sparse-read', False))
929 # sparse-revlog forces sparse-read
930 self._withsparseread = self._sparserevlog or withsparseread
927 if 'sparse-read-density-threshold' in opts: 931 if 'sparse-read-density-threshold' in opts:
928 self._srdensitythreshold = opts['sparse-read-density-threshold'] 932 self._srdensitythreshold = opts['sparse-read-density-threshold']
929 if 'sparse-read-min-gap-size' in opts: 933 if 'sparse-read-min-gap-size' in opts:
930 self._srmingapsize = opts['sparse-read-min-gap-size'] 934 self._srmingapsize = opts['sparse-read-min-gap-size']
931 935