mercurial-scm/hg-stable: tests/artifacts/scripts/generate-churning-bundle.py annotate

annotate tests/artifacts/scripts/generate-churning-bundle.py @ 53040:cdd7bf612c7b stable tip

bundle-spec: properly format boolean parameter (issue6960) This was breaking automatic clone bundle generation. This changeset fixes it and add a test to catch it in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 11 Mar 2025 02:29:42 +0100
parents	7e5ed1e80913
children

rev	line source
45849 c102b704edb5 global: use python3 in shebangs Gregory Szorc <gregory.szorc@gmail.com> parents: 43110 diff changeset	1 #!/usr/bin/env python3
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	2 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	3 # generate-branchy-bundle - generate a branch for a "large" branchy repository
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	4 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	5 # Copyright 2018 Octobus, contact@octobus.net
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	6 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	7 # This software may be used and distributed according to the terms of the
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	8 # GNU General Public License version 2 or any later version.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	9 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	10 # This script generates a repository suitable for testing delta computation
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	11 # strategies.
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	12 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	13 # The repository update a single "large" file with many updates. One fixed part
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	14 # of the files always get updated while the rest of the lines get updated over
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	15 # time. This update happens over many topological branches, some getting merged
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	16 # back.
52471 24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	17 #
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	18 # --lazy will skip generating the file if one exist with the right content
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	19 # already.
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	20 # --validate make sure the generated bundle has the expected content.
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	21
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	22
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	23 import hashlib
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	24 import os
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	25 import shutil
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	26 import subprocess
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	27 import sys
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	28 import tempfile
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	29
52470 83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	30 import mercurial.context
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	31 import mercurial.hg
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	32 import mercurial.ui
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	33
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	34 BUNDLE_NAME = 'big-file-churn.hg'
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	35
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	36 # constants for generating the repository
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	37 NB_CHANGESET = 5000
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	38 PERIOD_MERGING = 8
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	39 PERIOD_BRANCHING = 7
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	40 MOVE_BACK_MIN = 3
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	41 MOVE_BACK_RANGE = 5
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	42
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	43 # constants for generating the large file we keep updating
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	44 #
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	45 # At each revision, the beginning on the file change,
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	46 # and set of other lines changes too.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	47 FILENAME = 'SPARSE-REVLOG-TEST-FILE'
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	48 NB_LINES = 10500
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	49 ALWAYS_CHANGE_LINES = 500
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	50 OTHER_CHANGES = 300
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	51
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	52
52468 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	53 def build_graph():
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	54 heads = {0}
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	55 graph = {0: (None, None)}
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	56 for idx in range(1, NB_CHANGESET + 1):
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	57 p, _ = parents = [idx - 1, None]
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	58 if (idx % PERIOD_BRANCHING) == 0:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	59 back = MOVE_BACK_MIN + (idx % MOVE_BACK_RANGE)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	60 for _ in range(back):
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	61 p = graph.get(p, (p,))[0]
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	62 parents[0] = p
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	63 if (idx % PERIOD_MERGING) == 0:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	64 parents[1] = min(heads)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	65 for p in parents:
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	66 heads.discard(p)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	67 heads.add(idx)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	68 graph[idx] = tuple(parents)
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	69 return graph
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	70
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	71
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	72 GRAPH = build_graph()
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	73
e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	74
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	75 def nextcontent(previous_content):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	76 """utility to produce a new file content from the previous one"""
51152 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48966 diff changeset	77 return hashlib.md5(previous_content).hexdigest().encode('ascii')
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	78
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	79
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	80 def filecontent(iteridx, oldcontent):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	81 """generate a new file content
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	82
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	83 The content is generated according the iteration index and previous
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	84 content"""
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	85
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	86 # initial call
52468 e26b738430a1 test-sparse-revlog: pre-generate the graph structure in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52467 diff changeset	87 if iteridx == 0:
51152 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48966 diff changeset	88 current = b''
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	89 else:
51152 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48966 diff changeset	90 current = b"%d" % iteridx
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	91
43110 1fcf79e9943a tests: use range() in generate-churning-module.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	92 for idx in range(NB_LINES):
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	93 do_change_line = True
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	94 if oldcontent is not None and ALWAYS_CHANGE_LINES < idx:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	95 do_change_line = not ((idx - iteridx) % OTHER_CHANGES)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	96
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	97 if do_change_line:
51152 d7155949535e generate-churning-bundle: fix script for python3 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48966 diff changeset	98 to_write = current + b'\n'
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	99 current = nextcontent(current)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	100 else:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	101 to_write = oldcontent[idx]
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	102 yield to_write
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	103
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	104
52469 9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	105 def merge_content(base, left, right):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	106 """merge two file content to produce a new one
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	107
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	108 use unambiguous update on each side when possible, and produce a new line
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	109 whenever a merge is needed. Similar to what the manifest would do.
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	110 """
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	111 for old, left, right in zip(base, left, right):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	112 if old == left and old == right:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	113 yield old
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	114 elif old == left and old != right:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	115 yield right
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	116 elif old != left and old == right:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	117 yield left
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	118 else:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	119 yield nextcontent(left + right)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	120
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	121
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	122 def ancestors(graph, rev):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	123 """return the set of ancestors of revision <rev>"""
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	124 to_proceed = {rev}
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	125 seen = set(to_proceed)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	126 while to_proceed:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	127 current = to_proceed.pop()
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	128 for p in graph[current]:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	129 if p is None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	130 continue
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	131 if p in seen:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	132 continue
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	133 to_proceed.add(p)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	134 seen.add(p)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	135 return seen
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	136
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	137
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	138 def gca(graph, left, right):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	139 """find the greater common ancestors of left and right
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	140
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	141 Note that the algorithm is stupid and N² when run on all merge, however
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	142 this should not be a too much issue given the current scale.
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	143 """
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	144 return max(ancestors(graph, left) & ancestors(graph, right))
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	145
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	146
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	147 def make_one_content_fn(idx, base, left, right):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	148 """build a function that build the content on demand
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	149
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	150 The dependency are kept are reference to make sure they are not
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	151 garbage-collected until we use them. Once we computed the current content,
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	152 we make sure to drop their reference to allow them to be garbage collected.
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	153 """
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	154
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	155 def content_fn(idx=idx, base=base, left=left, right=right):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	156 if left is None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	157 new = filecontent(idx, None)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	158 elif base is None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	159 new = filecontent(idx, left())
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	160 else:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	161 merged = merge_content(base(), left(), right())
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	162 new = filecontent(idx, list(merged))
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	163 return list(new)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	164
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	165 del idx
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	166 del base
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	167 del left
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	168 del right
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	169
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	170 value = None
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	171 cf = [content_fn]
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	172 del content_fn
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	173
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	174 def final_fn():
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	175 nonlocal value
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	176 if value is None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	177 content_fn = cf.pop()
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	178 value = list(content_fn())
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	179 del content_fn
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	180 return value
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	181
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	182 return final_fn
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	183
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	184
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	185 def build_content_graph(graph):
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	186 """produce file content for all revision
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	187
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	188 The content will be generated on demande and cached. Cleanup the
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	189 dictionnary are you use it to reduce memory usage.
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	190 """
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	191 content = {}
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	192 for idx, (p1, p2) in graph.items():
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	193 base = left = right = None
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	194 if p1 is not None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	195 left = content[p1]
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	196 if p2 is not None:
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	197 right = content[p2]
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	198 base_rev = gca(graph, p1, p2)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	199 base = content[base_rev]
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	200 content[idx] = make_one_content_fn(idx, base, left, right)
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	201 return content
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	202
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	203
9feb175c028d test-sparse-revlog: build the content directly in memory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52468 diff changeset	204 CONTENT = build_content_graph(GRAPH)
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	205
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	206
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	207 def hg(command, *args):
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	208 """call a mercurial command with appropriate config and argument"""
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	209 env = os.environ.copy()
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	210 if 'CHGHG' in env:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	211 full_cmd = ['chg']
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	212 else:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	213 full_cmd = ['hg']
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	214 full_cmd.append('--quiet')
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	215 full_cmd.append(command)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	216 if command == 'commit':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	217 # reproducible commit metadata
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	218 full_cmd.extend(['--date', '0 0', '--user', 'test'])
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	219 elif command == 'merge':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	220 # avoid conflicts by picking the local variant
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	221 full_cmd.extend(['--tool', ':merge-local'])
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	222 full_cmd.extend(args)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	223 env['HGRCPATH'] = ''
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	224 return subprocess.check_call(full_cmd, env=env)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	225
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	226
52470 83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	227 def write_repo(path):
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	228 """write repository content in memory"""
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	229 repo = mercurial.hg.repository(
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	230 mercurial.ui.ui.load(),
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	231 path=path.encode('utf-8'),
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	232 )
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	233 nodemap = {None: repo.nodeconstants.nullid}
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	234 with repo.lock(), repo.transaction(b'bundle-generation'):
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	235 for idx, (p1, p2) in GRAPH.items():
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	236 if sys.stdout.isatty():
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	237 print("generating commit #%d/%d" % (idx, NB_CHANGESET))
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	238
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	239 file_fn = lambda repo, memctx, path: mercurial.context.memfilectx(
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	240 repo,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	241 memctx,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	242 path,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	243 data=b''.join(CONTENT.pop(idx)()),
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	244 )
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	245
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	246 mc = mercurial.context.memctx(
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	247 repo,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	248 (nodemap[p1], nodemap[p2]),
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	249 b'commit #%d' % idx if idx else b'initial commit',
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	250 [FILENAME.encode('ascii')],
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	251 file_fn,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	252 user=b"test",
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	253 date=(0, 0),
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	254 )
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	255 nodemap[idx] = repo.commitctx(mc)
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	256
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	257
52471 24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	258 def compute_md5(target):
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	259 with open(target, 'rb') as bundle:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	260 data = bundle.read()
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	261 return hashlib.md5(data).hexdigest()
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	262
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	263
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	264 def write_md5(target, md5):
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	265 with open(target + '.md5', 'wb') as md5file:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	266 md5file.write(md5.encode('ascii') + b'\n')
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	267
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	268
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	269 def read_md5(target):
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	270 with open(target + '.md5', 'rb') as md5file:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	271 return md5file.read().strip().decode('ascii')
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	272
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	273
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	274 def up_to_date_target(target):
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	275 """return true if the file already exist at the right"""
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	276 try:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	277 found = compute_md5(target)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	278 expected = read_md5(target)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	279 except OSError:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	280 return False
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	281 return found == expected
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	282
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	283
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	284 def run(target, validate=False):
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	285 tmpdir = tempfile.mkdtemp(prefix='tmp-hg-test-big-file-bundle-')
52483 7e5ed1e80913 generate-churning-bundle: change dir before deleting temp dir paugier <pierre.augier@univ-grenoble-alpes.fr> parents: 52471 diff changeset	286 cwd = os.getcwd()
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	287 try:
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	288 os.chdir(tmpdir)
52470 83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	289 hg(
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	290 'init',
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	291 '--config',
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	292 'format.maxchainlen=%d' % NB_CHANGESET,
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	293 )
83f87912c5e0 test-sparse-revlog: commit the repo content in a single in-mem transaction Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52469 diff changeset	294 write_repo(tmpdir)
41326 b59676077654 test: stabilize bundle generation for test-sparse-revlog.t Pierre-Yves David <pierre-yves.david@octobus.net> parents: 39508 diff changeset	295 hg('bundle', '--all', target, '--config', 'devel.bundle.delta=p1')
52471 24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	296 digest = compute_md5(target)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	297 if not validate:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	298 write_md5(target, digest)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	299 else:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	300 expected = read_md5(target)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	301 if expected != digest:
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	302 msg = "bundle generated does not match the expected content\n"
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	303 msg += " expected: %s\n" % expected
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	304 msg += " got: %s" % digest
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	305 print(msg, file=sys.stderr)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	306 return 1
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	307 finally:
52483 7e5ed1e80913 generate-churning-bundle: change dir before deleting temp dir paugier <pierre.augier@univ-grenoble-alpes.fr> parents: 52471 diff changeset	308 # Windows does not let you remove the current working directory
7e5ed1e80913 generate-churning-bundle: change dir before deleting temp dir paugier <pierre.augier@univ-grenoble-alpes.fr> parents: 52471 diff changeset	309 os.chdir(cwd)
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	310 shutil.rmtree(tmpdir)
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	311 return 0
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	312
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41805 diff changeset	313
39508 4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	314 if __name__ == '__main__':
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	315 orig = os.path.realpath(os.path.dirname(sys.argv[0]))
4ca7a67c94c8 sparse-revlog: add a test checking revlog deltas for a churning file Boris Feld <boris.feld@octobus.net> parents: diff changeset	316 target = os.path.join(orig, os.pardir, 'cache', BUNDLE_NAME)
52471 24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	317 lazy = '--lazy' in sys.argv[1:]
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	318 validate = '--validate' in sys.argv[1:]
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	319 if lazy and up_to_date_target(target):
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	320 sys.exit(0)
24c3b3dbab08 test-sparse-revlog: make the large bundle generation more robust and useful Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52470 diff changeset	321 sys.exit(run(target, validate=validate))

Mercurial > public > mercurial-scm > hg-stable

annotate tests/artifacts/scripts/generate-churning-bundle.py @ 53040:cdd7bf612c7b stable tip