annotate mercurial/utils/memorytop.py @ 51528:f85f23f1479b

branchcache: skip entries that are topological heads in the on disk file In the majority of cases, topological heads are also branch heads. We have efficient way to get the topological heads and efficient way to retrieve their branch information. So there is little value in putting them in the branch cache file explicitly. On the contrary, writing them explicitly tend to create very large cache file that are inefficient to read and update. So the branch cache v3 format is no longer including them. This changeset focus on the format aspect and have no focus on the performance aspect. We will cover that later.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Thu, 07 Mar 2024 10:55:22 +0100
parents 5b6c0af021da
children 1c5810ce737e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
45797
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
1 # memorytop requires Python 3.4
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
2 #
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
3 # Usage: set PYTHONTRACEMALLOC=n in the environment of the hg invocation,
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
4 # where n>= is the number of frames to show in the backtrace. Put calls to
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
5 # memorytop in strategic places to show the current memory use by allocation
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
6 # site.
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
7
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
8 import gc
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
9 import tracemalloc
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
10
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
11
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
12 def memorytop(limit=10):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
13 gc.collect()
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
14 snapshot = tracemalloc.take_snapshot()
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
15
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
16 snapshot = snapshot.filter_traces(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
17 (
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
18 tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
19 tracemalloc.Filter(False, "<frozen importlib._bootstrap_external>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
20 tracemalloc.Filter(False, "<unknown>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
21 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
22 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
23 stats = snapshot.statistics('traceback')
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
24
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
25 total = sum(stat.size for stat in stats)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
26 print("\nTotal allocated size: %.1f KiB\n" % (total / 1024))
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
27 print("Lines with the biggest net allocations")
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
28 for index, stat in enumerate(stats[:limit], 1):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
29 print(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
30 "#%d: %d objects using %.1f KiB"
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
31 % (index, stat.count, stat.size / 1024)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
32 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
33 for line in stat.traceback.format(most_recent_first=True):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
34 print(' ', line)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
35
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
36 other = stats[limit:]
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
37 if other:
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
38 size = sum(stat.size for stat in other)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
39 count = sum(stat.count for stat in other)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
40 print(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
41 "%s other: %d objects using %.1f KiB"
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
42 % (len(other), count, size / 1024)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
43 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
44 print()