annotate mercurial/cacheutil.py @ 48178:f12a19d03d2c

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280
author Danny Hooper <hooper@google.com>
date Thu, 02 Sep 2021 14:08:45 -0700
parents d4ba4d51f85f
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35766
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
1 # scmutil.py - Mercurial core utility functions
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 43077
diff changeset
3 # Copyright Olivia Mackall <olivia@selenic.com> and other
35766
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
4 #
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
7 from __future__ import absolute_import
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
8
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
9 from . import repoview
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
10
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 35766
diff changeset
11
35766
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
12 def cachetocopy(srcrepo):
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
13 """return the list of cache file valuable to copy during a clone"""
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
14 # In local clones we're copying all nodes, not just served
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
15 # ones. Therefore copy all branch caches over.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
16 cachefiles = [b'branch2']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
17 cachefiles += [b'branch2-%s' % f for f in repoview.filtertable]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
18 cachefiles += [b'rbc-names-v1', b'rbc-revs-v1']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
19 cachefiles += [b'tags2']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
20 cachefiles += [b'tags2-%s' % f for f in repoview.filtertable]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
21 cachefiles += [b'hgtagsfnodes1']
35766
72fdd99eb526 caches: make 'cachetocopy' available in scmutil
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
22 return cachefiles