mercurial-scm/hg: tests/artifacts/scripts/generate-churning-bundle.py annotate

annotate tests/artifacts/scripts/generate-churning-bundle.py @ 52437:e26b738430a1

test-sparse-revlog: pre-generate the graph structure in memory This is the first step toward a more "in memory" generation of that content that should be faster.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Wed, 04 Dec 2024 00:14:56 +0100
parents	599c696bb514
children	9feb175c028d

rev	line source
45830 c102b704edb5 global: use python3 in shebangs Gregory Szorc <gregory.szorc@gmail.com> parents: 43110 diff changeset	1 #!/usr/bin/env python3
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	2 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	3 # generate-branchy-bundle - generate a branch for a "large" branchy repository
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	4 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	5 # Copyright 2018 Octobus, contact@octobus.net
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	6 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	7 # This software may be used and distributed according to the terms of the
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	8 # GNU General Public License version 2 or any later version.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	9 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	10 # This script generates a repository suitable for testing delta computation
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	11 # strategies.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	12 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	13 # The repository update a single "large" file with many updates. One fixed part
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	14 # of the files always get updated while the rest of the lines get updated over
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	15 # time. This update happens over many topological branches, some getting merged
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	16 # back.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	17 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	18 # Running with `chg` in your path and `CHGHG` set is recommended for speed.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	19
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	20
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	21 import hashlib
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	22 import os
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	23 import shutil
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	24 import subprocess
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	25 import sys
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	26 import tempfile
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	27
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	28 BUNDLE_NAME = 'big-file-churn.hg'
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	29
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	30 # constants for generating the repository
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	31 NB_CHANGESET = 5000
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	32 PERIOD_MERGING = 8
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	33 PERIOD_BRANCHING = 7
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	34 MOVE_BACK_MIN = 3
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	35 MOVE_BACK_RANGE = 5
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	36
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	37 # constants for generating the large file we keep updating
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	38 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	39 # At each revision, the beginning on the file change,
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	40 # and set of other lines changes too.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	41 FILENAME = 'SPARSE-REVLOG-TEST-FILE'
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	42 NB_LINES = 10500
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	43 ALWAYS_CHANGE_LINES = 500
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	44 OTHER_CHANGES = 300
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	45
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	46
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	47 def build_graph():
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	48 heads = {0}
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	49 graph = {0: (None, None)}
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	50 for idx in range(1, NB_CHANGESET + 1):
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	51 p, _ = parents = [idx - 1, None]
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	52 if (idx % PERIOD_BRANCHING) == 0:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	53 back = MOVE_BACK_MIN + (idx % MOVE_BACK_RANGE)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	54 for _ in range(back):
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	55 p = graph.get(p, (p,))[0]
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	56 parents[0] = p
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	57 if (idx % PERIOD_MERGING) == 0:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	58 parents[1] = min(heads)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	59 for p in parents:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	60 heads.discard(p)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	61 heads.add(idx)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	62 graph[idx] = tuple(parents)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	63 return graph
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	64
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	65
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	66 GRAPH = build_graph()
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	67
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	68
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	69 def nextcontent(previous_content):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	70 """utility to produce a new file content from the previous one"""
51274 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48875 diff changeset	71 return hashlib.md5(previous_content).hexdigest().encode('ascii')
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	72
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	73
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	74 def filecontent(iteridx, oldcontent):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	75 """generate a new file content
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	76
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	77 The content is generated according the iteration index and previous
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	78 content"""
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	79
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	80 # initial call
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	81 if iteridx == 0:
51274 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48875 diff changeset	82 current = b''
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	83 else:
51274 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48875 diff changeset	84 current = b"%d" % iteridx
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	85
43110 1fcf79e9943a tests: use range() in generate-churning-module.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	86 for idx in range(NB_LINES):
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	87 do_change_line = True
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	88 if oldcontent is not None and ALWAYS_CHANGE_LINES < idx:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	89 do_change_line = not ((idx - iteridx) % OTHER_CHANGES)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	90
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	91 if do_change_line:
51274 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48875 diff changeset	92 to_write = current + b'\n'
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	93 current = nextcontent(current)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	94 else:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	95 to_write = oldcontent[idx]
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	96 yield to_write
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	97
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	98
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	99 def updatefile(filename, idx):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	100 """update <filename> to be at appropriate content for iteration <idx>"""
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	101 existing = None
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	102 if idx > 0:
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	103 with open(filename, 'rb') as old:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	104 existing = old.readlines()
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	105 with open(filename, 'wb') as target:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	106 for line in filecontent(idx, existing):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	107 target.write(line)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	108
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	109
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	110 def hg(command, *args):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	111 """call a mercurial command with appropriate config and argument"""
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	112 env = os.environ.copy()
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	113 if 'CHGHG' in env:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	114 full_cmd = ['chg']
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	115 else:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	116 full_cmd = ['hg']
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	117 full_cmd.append('--quiet')
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	118 full_cmd.append(command)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	119 if command == 'commit':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	120 # reproducible commit metadata
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	121 full_cmd.extend(['--date', '0 0', '--user', 'test'])
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	122 elif command == 'merge':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	123 # avoid conflicts by picking the local variant
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	124 full_cmd.extend(['--tool', ':merge-local'])
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	125 full_cmd.extend(args)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	126 env['HGRCPATH'] = ''
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	127 return subprocess.check_call(full_cmd, env=env)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	128
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	129
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	130 def run(target):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	131 tmpdir = tempfile.mkdtemp(prefix='tmp-hg-test-big-file-bundle-')
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	132 try:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	133 os.chdir(tmpdir)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	134 hg('init')
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	135 for idx, (p1, p2) in GRAPH.items():
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	136 if sys.stdout.isatty():
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	137 print("generating commit #%d/%d" % (idx, NB_CHANGESET))
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	138 if p1 is not None and p1 != idx - 1:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	139 hg('update', "%d" % p1)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	140 if p2 is not None:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	141 hg('merge', "%d" % p2)
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	142 updatefile(FILENAME, idx)
52437 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	143 if idx == 0:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	144 hg('add', FILENAME)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	145 hg('commit', '--addremove', '--message', 'initial commit')
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	146 else:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52436 diff changeset	147 hg('commit', '--message', 'commit #%d' % idx)
41795 b59676077654 test: stabilize bundle generation for test-sparse-revlog.t Pierre-Yves David <pierre-yves.david@octobus.net> parents: 39491 diff changeset	148 hg('bundle', '--all', target, '--config', 'devel.bundle.delta=p1')
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	149 with open(target, 'rb') as bundle:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	150 data = bundle.read()
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	151 digest = hashlib.md5(data).hexdigest()
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	152 with open(target + '.md5', 'wb') as md5file:
51274 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48875 diff changeset	153 md5file.write(digest.encode('ascii') + b'\n')
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	154 if sys.stdout.isatty():
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	155 print('bundle generated at "%s" md5: %s' % (target, digest))
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	156
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	157 finally:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	158 shutil.rmtree(tmpdir)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	159 return 0
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	160
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41800 diff changeset	161
39491 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	162 if __name__ == '__main__':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	163 orig = os.path.realpath(os.path.dirname(sys.argv[0]))
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	164 target = os.path.join(orig, os.pardir, 'cache', BUNDLE_NAME)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	165 sys.exit(run(target))

Mercurial > public > mercurial-scm > hg

annotate tests/artifacts/scripts/generate-churning-bundle.py @ 52437:e26b738430a1