Mercurial > public > mercurial-scm > hg

comparison mercurial/localrepo.py @ 39700:b10d145837bc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
localrepo: extract resolving of opener options to standalone functions Requirements and config options are converted into a dict which is available to the store vfs to consult. This is how storage options are communicated from the repo layer to the storage layer. Currently, we do that option resolution in a private method on the repo instance. And there is a single method doing that resolution. Opener options are logically specific to the storage backend they apply to. And, opener options may wish to influence how the repo object/type is constructed. So it makes sense to have more granular storage option resolution that occurs before the repo object is instantiated. This commit extracts the code for resolving opener options into new module-level functions. These functions are run before the repo instance is constructed. As part of the code move, we split the option resolution into generic and revlog-specific options. After this commit, we no longer add revlog-specific options to repos that don't have a revlog requirement. Some of these opener options and associated config options might make sense on alternate storage backends. We can always reuse config options and opener option names for other backends. But we shouldn't be passing opener options to storage backends that won't recognize them. I haven't done it here, but after this commit it should be possible for store backends to validate the set of opener options it receives. Because localrepository.openerreqs is no longer used after this commit, it has been removed. I'm not super thrilled about the code outside of localrepo that is adding requirements and updating opener options. We'll probably want to create a more formal API for that use case that constructs a new repo instance and poisons the old repo object. But this was a pre-existing issue and can be dealt with later. I have little doubt it will cause me troubles as I continue to refactor how repository objects are instantiated. .. api:: ``localrepository.openerreqs`` has been removed. Override ``localrepo.resolvestorevfsoptions()`` to add custom opener options. .. api:: ``localrepository._applyopenerreqs()`` has been removed. Use ``localrepo.resolvestorevfsoptions()`` to add custom opener options. Differential Revision: https://phab.mercurial-scm.org/D4576
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 12 Sep 2018 15:59:26 -0700
parents 6f26417b71bb
children 543f26ece6cf
comparison
equal deleted inserted replaced
39699:6f26417b71bb 39700:b10d145837bc
479 # The store has changed over time and the exact layout is dictated by 479 # The store has changed over time and the exact layout is dictated by
480 # requirements. The store interface abstracts differences across all 480 # requirements. The store interface abstracts differences across all
481 # of them. 481 # of them.
482 store = makestore(requirements, storebasepath, 482 store = makestore(requirements, storebasepath,
483 lambda base: vfsmod.vfs(base, cacheaudited=True)) 483 lambda base: vfsmod.vfs(base, cacheaudited=True))
484
485 hgvfs.createmode = store.createmode 484 hgvfs.createmode = store.createmode
485
486 storevfs = store.vfs
487 storevfs.options = resolvestorevfsoptions(ui, requirements)
486 488
487 # The cache vfs is used to manage cache files. 489 # The cache vfs is used to manage cache files.
488 cachevfs = vfsmod.vfs(cachepath, cacheaudited=True) 490 cachevfs = vfsmod.vfs(cachepath, cacheaudited=True)
489 cachevfs.createmode = store.createmode 491 cachevfs.createmode = store.createmode
490 492
575 b'dotencode' in requirements) 577 b'dotencode' in requirements)
576 578
577 return storemod.encodedstore(path, vfstype) 579 return storemod.encodedstore(path, vfstype)
578 580
579 return storemod.basicstore(path, vfstype) 581 return storemod.basicstore(path, vfstype)
582
583 def resolvestorevfsoptions(ui, requirements):
584 """Resolve the options to pass to the store vfs opener.
585
586 The returned dict is used to influence behavior of the storage layer.
587 """
588 options = {}
589
590 if b'treemanifest' in requirements:
591 options[b'treemanifest'] = True
592
593 # experimental config: format.manifestcachesize
594 manifestcachesize = ui.configint(b'format', b'manifestcachesize')
595 if manifestcachesize is not None:
596 options[b'manifestcachesize'] = manifestcachesize
597
598 # In the absence of another requirement superseding a revlog-related
599 # requirement, we have to assume the repo is using revlog version 0.
600 # This revlog format is super old and we don't bother trying to parse
601 # opener options for it because those options wouldn't do anything
602 # meaningful on such old repos.
603 if b'revlogv1' in requirements or REVLOGV2_REQUIREMENT in requirements:
604 options.update(resolverevlogstorevfsoptions(ui, requirements))
605
606 return options
607
608 def resolverevlogstorevfsoptions(ui, requirements):
609 """Resolve opener options specific to revlogs."""
610
611 options = {}
612
613 if b'revlogv1' in requirements:
614 options[b'revlogv1'] = True
615 if REVLOGV2_REQUIREMENT in requirements:
616 options[b'revlogv2'] = True
617
618 if b'generaldelta' in requirements:
619 options[b'generaldelta'] = True
620
621 # experimental config: format.chunkcachesize
622 chunkcachesize = ui.configint(b'format', b'chunkcachesize')
623 if chunkcachesize is not None:
624 options[b'chunkcachesize'] = chunkcachesize
625
626 deltabothparents = ui.configbool(b'storage',
627 b'revlog.optimize-delta-parent-choice')
628 options[b'deltabothparents'] = deltabothparents
629
630 options[b'lazydeltabase'] = not scmutil.gddeltaconfig(ui)
631
632 chainspan = ui.configbytes(b'experimental', b'maxdeltachainspan')
633 if 0 <= chainspan:
634 options[b'maxdeltachainspan'] = chainspan
635
636 mmapindexthreshold = ui.configbytes(b'experimental',
637 b'mmapindexthreshold')
638 if mmapindexthreshold is not None:
639 options[b'mmapindexthreshold'] = mmapindexthreshold
640
641 withsparseread = ui.configbool(b'experimental', b'sparse-read')
642 srdensitythres = float(ui.config(b'experimental',
643 b'sparse-read.density-threshold'))
644 srmingapsize = ui.configbytes(b'experimental',
645 b'sparse-read.min-gap-size')
646 options[b'with-sparse-read'] = withsparseread
647 options[b'sparse-read-density-threshold'] = srdensitythres
648 options[b'sparse-read-min-gap-size'] = srmingapsize
649
650 sparserevlog = SPARSEREVLOG_REQUIREMENT in requirements
651 options[b'sparse-revlog'] = sparserevlog
652 if sparserevlog:
653 options[b'generaldelta'] = True
654
655 maxchainlen = None
656 if sparserevlog:
657 maxchainlen = revlogconst.SPARSE_REVLOG_MAX_CHAIN_LENGTH
658 # experimental config: format.maxchainlen
659 maxchainlen = ui.configint(b'format', b'maxchainlen', maxchainlen)
660 if maxchainlen is not None:
661 options[b'maxchainlen'] = maxchainlen
662
663 for r in requirements:
664 if r.startswith(b'exp-compression-'):
665 options[b'compengine'] = r[len(b'exp-compression-'):]
666
667 return options
580 668
581 @interfaceutil.implementer(repository.completelocalrepository) 669 @interfaceutil.implementer(repository.completelocalrepository)
582 class localrepository(object): 670 class localrepository(object):
583 671
584 # obsolete experimental requirements: 672 # obsolete experimental requirements:
600 'relshared', 688 'relshared',
601 'dotencode', 689 'dotencode',
602 'exp-sparse', 690 'exp-sparse',
603 'internal-phase' 691 'internal-phase'
604 } 692 }
605 openerreqs = {
606 'revlogv1',
607 'generaldelta',
608 'treemanifest',
609 }
610 693
611 # list of prefix for file which can be written without 'wlock' 694 # list of prefix for file which can be written without 'wlock'
612 # Extensions should extend this list when needed 695 # Extensions should extend this list when needed
613 _wlockfreeprefix = { 696 _wlockfreeprefix = {
614 # We migh consider requiring 'wlock' for the next 697 # We migh consider requiring 'wlock' for the next
710 self.ui.configbool('devel', 'check-locks')): 793 self.ui.configbool('devel', 'check-locks')):
711 if util.safehasattr(self.svfs, 'vfs'): # this is filtervfs 794 if util.safehasattr(self.svfs, 'vfs'): # this is filtervfs
712 self.svfs.vfs.audit = self._getsvfsward(self.svfs.vfs.audit) 795 self.svfs.vfs.audit = self._getsvfsward(self.svfs.vfs.audit)
713 else: # standard vfs 796 else: # standard vfs
714 self.svfs.audit = self._getsvfsward(self.svfs.audit) 797 self.svfs.audit = self._getsvfsward(self.svfs.audit)
715 self._applyopenerreqs()
716 798
717 self._dirstatevalidatewarned = False 799 self._dirstatevalidatewarned = False
718 800
719 self._branchcaches = {} 801 self._branchcaches = {}
720 self._revbranchcache = None 802 self._revbranchcache = None
814 caps = set(caps) 896 caps = set(caps)
815 capsblob = bundle2.encodecaps(bundle2.getrepocaps(self, 897 capsblob = bundle2.encodecaps(bundle2.getrepocaps(self,
816 role='client')) 898 role='client'))
817 caps.add('bundle2=' + urlreq.quote(capsblob)) 899 caps.add('bundle2=' + urlreq.quote(capsblob))
818 return caps 900 return caps
819
820 def _applyopenerreqs(self):
821 self.svfs.options = {r: True for r in self.requirements
822 if r in self.openerreqs}
823 # experimental config: format.chunkcachesize
824 chunkcachesize = self.ui.configint('format', 'chunkcachesize')
825 if chunkcachesize is not None:
826 self.svfs.options['chunkcachesize'] = chunkcachesize
827 # experimental config: format.manifestcachesize
828 manifestcachesize = self.ui.configint('format', 'manifestcachesize')
829 if manifestcachesize is not None:
830 self.svfs.options['manifestcachesize'] = manifestcachesize
831 deltabothparents = self.ui.configbool('storage',
832 'revlog.optimize-delta-parent-choice')
833 self.svfs.options['deltabothparents'] = deltabothparents
834 self.svfs.options['lazydeltabase'] = not scmutil.gddeltaconfig(self.ui)
835 chainspan = self.ui.configbytes('experimental', 'maxdeltachainspan')
836 if 0 <= chainspan:
837 self.svfs.options['maxdeltachainspan'] = chainspan
838 mmapindexthreshold = self.ui.configbytes('experimental',
839 'mmapindexthreshold')
840 if mmapindexthreshold is not None:
841 self.svfs.options['mmapindexthreshold'] = mmapindexthreshold
842 withsparseread = self.ui.configbool('experimental', 'sparse-read')
843 srdensitythres = float(self.ui.config('experimental',
844 'sparse-read.density-threshold'))
845 srmingapsize = self.ui.configbytes('experimental',
846 'sparse-read.min-gap-size')
847 self.svfs.options['with-sparse-read'] = withsparseread
848 self.svfs.options['sparse-read-density-threshold'] = srdensitythres
849 self.svfs.options['sparse-read-min-gap-size'] = srmingapsize
850 sparserevlog = SPARSEREVLOG_REQUIREMENT in self.requirements
851 self.svfs.options['sparse-revlog'] = sparserevlog
852 if sparserevlog:
853 self.svfs.options['generaldelta'] = True
854 maxchainlen = None
855 if sparserevlog:
856 maxchainlen = revlogconst.SPARSE_REVLOG_MAX_CHAIN_LENGTH
857 # experimental config: format.maxchainlen
858 maxchainlen = self.ui.configint('format', 'maxchainlen', maxchainlen)
859 if maxchainlen is not None:
860 self.svfs.options['maxchainlen'] = maxchainlen
861
862 for r in self.requirements:
863 if r.startswith('exp-compression-'):
864 self.svfs.options['compengine'] = r[len('exp-compression-'):]
865
866 # TODO move "revlogv2" to openerreqs once finalized.
867 if REVLOGV2_REQUIREMENT in self.requirements:
868 self.svfs.options['revlogv2'] = True
869 901
870 def _writerequirements(self): 902 def _writerequirements(self):
871 scmutil.writerequires(self.vfs, self.requirements) 903 scmutil.writerequires(self.vfs, self.requirements)
872 904
873 # Don't cache auditor/nofsauditor, or you'll end up with reference cycle: 905 # Don't cache auditor/nofsauditor, or you'll end up with reference cycle: