Mercurial > public > mercurial-scm > hg-stable

comparison mercurial/localrepo.py @ 38717:aa21a9ad46ea

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
sparse-revlog: new requirement enabled with format.sparse-revlog The meaning of the new 'sparse-revlog' requirement is that the revlogs are allowed to contain wider delta chains with larger holes between the interesting chunks. These sparse delta chains should be read in several chunks to avoid a potential explosion of memory usage. Former version won't know how to read a delta chain in several chunks. They would keep reading them in a single read, and therefore would be subject to the potential memory explosion. Hence this new requirement: only versions having support of sparse-revlog reading should be allowed to read such a revlog. Implementation of this new algorithm and tools to enable or disable the requirement will follow in the next changesets.
author Paul Morelle <paul.morelle@octobus.net>
date Mon, 04 Jun 2018 22:23:18 +0200
parents ead71b15efd5
children 93777d16a25d
comparison
equal deleted inserted replaced
38716:c67093e81a3e 38717:aa21a9ad46ea
352 352
353 # Increment the sub-version when the revlog v2 format changes to lock out old 353 # Increment the sub-version when the revlog v2 format changes to lock out old
354 # clients. 354 # clients.
355 REVLOGV2_REQUIREMENT = 'exp-revlogv2.0' 355 REVLOGV2_REQUIREMENT = 'exp-revlogv2.0'
356 356
357 # A repository with the sparserevlog feature will have delta chains that
358 # can spread over a larger span. Sparse reading cuts these large spans into
359 # pieces, so that each piece isn't too big.
360 # Without the sparserevlog capability, reading from the repository could use
361 # huge amounts of memory, because the whole span would be read at once,
362 # including all the intermediate revisions that aren't pertinent for the chain.
363 # This is why once a repository has enabled sparse-read, it becomes required.
364 SPARSEREVLOG_REQUIREMENT = 'sparserevlog'
365
357 # Functions receiving (ui, features) that extensions can register to impact 366 # Functions receiving (ui, features) that extensions can register to impact
358 # the ability to load repositories with custom requirements. Only 367 # the ability to load repositories with custom requirements. Only
359 # functions defined in loaded extensions are called. 368 # functions defined in loaded extensions are called.
360 # 369 #
361 # The function receives a set of requirement strings that the repository 370 # The function receives a set of requirement strings that the repository
374 supportedformats = { 383 supportedformats = {
375 'revlogv1', 384 'revlogv1',
376 'generaldelta', 385 'generaldelta',
377 'treemanifest', 386 'treemanifest',
378 REVLOGV2_REQUIREMENT, 387 REVLOGV2_REQUIREMENT,
388 SPARSEREVLOG_REQUIREMENT,
379 } 389 }
380 _basesupported = supportedformats | { 390 _basesupported = supportedformats | {
381 'store', 391 'store',
382 'fncache', 392 'fncache',
383 'shared', 393 'shared',
676 srmingapsize = self.ui.configbytes('experimental', 686 srmingapsize = self.ui.configbytes('experimental',
677 'sparse-read.min-gap-size') 687 'sparse-read.min-gap-size')
678 self.svfs.options['with-sparse-read'] = withsparseread 688 self.svfs.options['with-sparse-read'] = withsparseread
679 self.svfs.options['sparse-read-density-threshold'] = srdensitythres 689 self.svfs.options['sparse-read-density-threshold'] = srdensitythres
680 self.svfs.options['sparse-read-min-gap-size'] = srmingapsize 690 self.svfs.options['sparse-read-min-gap-size'] = srmingapsize
691 sparserevlog = SPARSEREVLOG_REQUIREMENT in self.requirements
692 self.svfs.options['sparse-revlog'] = sparserevlog
681 693
682 for r in self.requirements: 694 for r in self.requirements:
683 if r.startswith('exp-compression-'): 695 if r.startswith('exp-compression-'):
684 self.svfs.options['compengine'] = r[len('exp-compression-'):] 696 self.svfs.options['compengine'] = r[len('exp-compression-'):]
685 697
2368 2380
2369 if scmutil.gdinitconfig(ui): 2381 if scmutil.gdinitconfig(ui):
2370 requirements.add('generaldelta') 2382 requirements.add('generaldelta')
2371 if ui.configbool('experimental', 'treemanifest'): 2383 if ui.configbool('experimental', 'treemanifest'):
2372 requirements.add('treemanifest') 2384 requirements.add('treemanifest')
2385 # experimental config: format.sparse-revlog
2386 if ui.configbool('format', 'sparse-revlog'):
2387 requirements.add(SPARSEREVLOG_REQUIREMENT)
2373 2388
2374 revlogv2 = ui.config('experimental', 'revlogv2') 2389 revlogv2 = ui.config('experimental', 'revlogv2')
2375 if revlogv2 == 'enable-unstable-format-and-corrupt-my-data': 2390 if revlogv2 == 'enable-unstable-format-and-corrupt-my-data':
2376 requirements.remove('revlogv1') 2391 requirements.remove('revlogv1')
2377 # generaldelta is implied by revlogv2. 2392 # generaldelta is implied by revlogv2.