mercurial-scm/hg: contrib/shrink-revlog.py annotate

annotate contrib/shrink-revlog.py @ 10623:64e286c22f29

shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Based on a patch by Benoit Boissinot, adapted to the pluggable sort algorithm design. toposort_reversepostorder() is a very good performer; it's designed to recreate what the revlog would have looked like if Mercurial had parent deltas now. toposort_postorderreverse() is unstable and very inconsistent, but perhaps with some work it could be made better.

author	Greg Ward <greg-hg@gerg.ca>
date	Tue, 09 Mar 2010 21:22:01 -0500
parents	bc81f126139f
children	432eb853a2c6

rev	line source
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	1 #!/usr/bin/env python
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	2
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	3 """\
10236 49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	4 reorder a revlog (the manifest by default) to save space
49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	5
49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	6 Specifically, this topologically sorts the revisions in the revlog so that
49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	7 revisions on the same branch are adjacent as much as possible. This is a
49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	8 workaround for the fact that Mercurial computes deltas relative to the
10216 843f6ee6d14b contrib: small documentation fixes in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10215 diff changeset	9 previous revision rather than relative to a parent revision.
843f6ee6d14b contrib: small documentation fixes in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10215 diff changeset	10
843f6ee6d14b contrib: small documentation fixes in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10215 diff changeset	11 This is not safe to run on a changelog.
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	12 """
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	13
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	14 # Originally written by Benoit Boissinot <benoit.boissinot at ens-lyon.org>
10216 843f6ee6d14b contrib: small documentation fixes in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10215 diff changeset	15 # as a patch to rewrite-log. Cleaned up, refactored, documented, and
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	16 # renamed by Greg Ward <greg at gerg.ca>.
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	17
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	18 # XXX would be nice to have a way to verify the repository after shrinking,
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	19 # e.g. by comparing "before" and "after" states of random changesets
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	20 # (maybe: export before, shrink, export after, diff).
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	21
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	22 import os, tempfile, errno
10509 3e7e789d9494 shrink-revlog: remove unneeded imports and useless code Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10508 diff changeset	23 from mercurial import revlog, transaction, node, util
10009 69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	24 from mercurial import changegroup
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	25 from mercurial.i18n import _
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	26
10622 bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	27 def toposort_branchsort(ui, rl):
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	28
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	29 children = {}
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	30 root = []
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	31 # build children and roots
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	32 ui.status(_('reading revs\n'))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	33 try:
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	34 for rev in rl:
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	35 ui.progress(_('reading'), rev, total=len(rl))
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	36 children[rev] = []
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	37 parents = [p for p in rl.parentrevs(rev) if p != node.nullrev]
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	38 # in case of duplicate parents
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	39 if len(parents) == 2 and parents[0] == parents[1]:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	40 del parents[1]
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	41 for p in parents:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	42 assert p in children
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	43 children[p].append(rev)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	44
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	45 if len(parents) == 0:
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	46 root.append(rev)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	47 finally:
10496 45734b51c99b progress: mark strings for translation Martin Geisler <mg@lazybytes.net> parents: 10440 diff changeset	48 ui.progress(_('reading'), None, total=len(rl))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	49
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	50 # XXX this is a reimplementation of the 'branchsort' topo sort
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	51 # algorithm in hgext.convert.convcmd... would be nice not to duplicate
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	52 # the algorithm
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	53 ui.status(_('sorting revs\n'))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	54 visit = root
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	55 result = []
10620 1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	56
1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	57 # suboptimal: nodes whose predecessor is not first parent
1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	58 suboptimal = 0
1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	59
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	60 while visit:
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	61 cur = visit.pop(0)
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	62 # revlog will compute delta relative to result[-1], so keep track
10620 1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	63 # of nodes where this might result in a large delta
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	64 parents = rl.parentrevs(cur)
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	65 if result:
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	66 if result[-1] != parents[0]:
10620 1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	67 suboptimal += 1
1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	68
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	69 result.append(cur)
c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	70 if cur not in children:
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	71 # This only happens if some node's p1 == p2, which can
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	72 # happen in the manifest in certain circumstances.
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	73 continue
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	74 next = []
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	75 for c in children.pop(cur):
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	76 parents_unseen = [p for p in rl.parentrevs(c)
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	77 if p != node.nullrev and p in children]
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	78 if len(parents_unseen) == 0:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	79 next.append(c)
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	80 visit = next + visit
10620 1ee14abe07b4 shrink-revlog: instrument sort code to record statistics. Greg Ward <greg-hg@gerg.ca> parents: 10542 diff changeset	81 ui.note(_('%d suboptimal nodes\n') % suboptimal)
10621 c93f0a23f381 shrink-revlog: rename some local variables for consistency. Greg Ward <greg-hg@gerg.ca> parents: 10620 diff changeset	82 return result
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	83
10623 64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	84 def toposort_reversepostorder(ui, rl):
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	85 # postorder of the reverse directed graph
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	86
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	87 # map rev to list of parent revs (p2 first)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	88 parents = {}
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	89 heads = set()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	90 ui.status(_('reading revs\n'))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	91 try:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	92 for rev in rl:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	93 ui.progress(_('reading'), rev, total=len(rl))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	94 (p1, p2) = rl.parentrevs(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	95 if p1 == p2 == node.nullrev:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	96 parents[rev] = () # root node
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	97 elif p1 == p2 or p2 == node.nullrev:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	98 parents[rev] = (p1,) # normal node
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	99 else:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	100 parents[rev] = (p2, p1) # merge node
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	101
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	102 heads.add(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	103
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	104 for p in parents[rev]:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	105 heads.discard(p)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	106 finally:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	107 ui.progress(_('reading'), None, total=len(rl))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	108
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	109 ui.status(_('sorting revs\n'))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	110 result = []
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	111 visit = list(heads)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	112 visit.sort(reverse=True)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	113 finished = set()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	114
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	115 while visit:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	116 cur = visit[-1]
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	117 for p in parents[cur]:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	118 if p not in finished:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	119 visit.append(p)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	120 break
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	121 else:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	122 result.append(cur)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	123 finished.add(cur)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	124 visit.pop()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	125 return result
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	126
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	127 def toposort_postorderreverse(ui, rl):
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	128 # reverse-postorder of the reverse directed graph
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	129
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	130 children = {}
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	131 roots = set()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	132 ui.status(_('reading revs\n'))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	133 try:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	134 for rev in rl:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	135 ui.progress(_('reading'), rev, total=len(rl))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	136 (p1, p2) = rl.parentrevs(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	137 if p1 == p2 == node.nullrev:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	138 roots.add(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	139
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	140 children[rev] = []
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	141
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	142 if p1 != node.nullrev:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	143 children[p1].append(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	144 if p2 != node.nullrev:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	145 children[p2].append(rev)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	146 finally:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	147 ui.progress(_('reading'), None, total=len(rl))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	148
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	149 ui.status(_('sorting revs\n'))
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	150 result = []
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	151 visit = list(roots)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	152 visit.sort()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	153 finished = set()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	154 while visit:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	155 cur = visit[-1]
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	156 for p in children[cur]:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	157 if p not in finished:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	158 visit.append(p)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	159 break
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	160 else:
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	161 result.append(cur)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	162 finished.add(cur)
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	163 visit.pop()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	164 result.reverse()
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	165 return result
64e286c22f29 shrink-revlog: add "reverse postorder" and "postorder reverse" toposorts. Greg Ward <greg-hg@gerg.ca> parents: 10622 diff changeset	166
10213 9e6848f352b0 contrib: use ui to write in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10009 diff changeset	167 def writerevs(ui, r1, r2, order, tr):
10009 69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	168
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	169 ui.status(_('writing revs\n'))
10440 b39b32c33269 shrink: use progress API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10282 diff changeset	170
10009 69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	171 count = [0]
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	172 def progress(*args):
10496 45734b51c99b progress: mark strings for translation Martin Geisler <mg@lazybytes.net> parents: 10440 diff changeset	173 ui.progress(_('writing'), count[0], total=len(order))
10009 69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	174 count[0] += 1
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	175
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	176 order = [r1.node(r) for r in order]
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	177
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	178 # this is a bit ugly, but it works
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	179 lookup = lambda x: "%020d" % r1.linkrev(r1.rev(x))
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	180 unlookup = lambda x: int(x, 10)
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	181
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	182 try:
10009 69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	183 group = util.chunkbuffer(r1.group(order, lookup, progress))
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	184 chunkiter = changegroup.chunkiter(group)
69dca8574a6a shrink-revlog: improve performance: use changegroup instead of revisions Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 9712 diff changeset	185 r2.addgroup(chunkiter, unlookup, tr)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	186 finally:
10496 45734b51c99b progress: mark strings for translation Martin Geisler <mg@lazybytes.net> parents: 10440 diff changeset	187 ui.progress(_('writing'), None, len(order))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	188
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	189 def report(ui, r1, r2):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	190 def getsize(r):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	191 s = 0
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	192 for fn in (r.indexfile, r.datafile):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	193 try:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	194 s += os.stat(fn).st_size
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	195 except OSError, inst:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	196 if inst.errno != errno.ENOENT:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	197 raise
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	198 return s
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	199
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	200 oldsize = float(getsize(r1))
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	201 newsize = float(getsize(r2))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	202
9712 18b134ef294c kill trailing whitespace Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 9515 diff changeset	203 # argh: have to pass an int to %d, because a float >= 2^32
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	204 # blows up under Python 2.5 or earlier
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	205 ui.write(_('old file size: %12d bytes (%6.1f MiB)\n')
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10241 diff changeset	206 % (int(oldsize), oldsize / 1024 / 1024))
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	207 ui.write(_('new file size: %12d bytes (%6.1f MiB)\n')
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10241 diff changeset	208 % (int(newsize), newsize / 1024 / 1024))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	209
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	210 shrink_percent = (oldsize - newsize) / oldsize * 100
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	211 shrink_factor = oldsize / newsize
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	212 ui.write(_('shrinkage: %.1f%% (%.1fx)\n')
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	213 % (shrink_percent, shrink_factor))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	214
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	215 def shrink(ui, repo, **opts):
10622 bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	216 """shrink a revlog by reordering revisions
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	217
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	218 Rewrites all the entries in some revlog of the current repository
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	219 (by default, the manifest log) to save space.
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	220
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	221 Different sort algorithms have different performance
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	222 characteristics. Use ``--sort`` to select a sort algorithm so you
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	223 can determine which works best for your data. The default
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	224 algorithm, ``branchsort``, works well for workflows with lots of
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	225 active (unmerged) branches, but not so well when all branches have
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	226 been merged and there is only one repository head.
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	227 """
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	228
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	229 if not repo.local():
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	230 raise util.Abort(_('not a local repository: %s') % repo.root)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	231
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	232 fn = opts.get('revlog')
9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	233 if not fn:
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	234 indexfn = repo.sjoin('00manifest.i')
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	235 else:
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	236 if not fn.endswith('.i'):
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	237 raise util.Abort(_('--revlog option must specify the revlog index '
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	238 'file (*.i), not %s') % opts.get('revlog'))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	239
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	240 indexfn = os.path.realpath(fn)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	241 store = repo.sjoin('')
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	242 if not indexfn.startswith(store):
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	243 raise util.Abort(_('--revlog option must specify a revlog in %s, '
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	244 'not %s') % (store, indexfn))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	245
10622 bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	246 sortname = opts['sort']
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	247 try:
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	248 toposort = globals()['toposort_' + sortname]
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	249 except KeyError:
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	250 raise util.Abort(_('no such toposort algorithm: %s') % sortname)
bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	251
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	252 if not os.path.exists(indexfn):
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	253 raise util.Abort(_('no such file: %s') % indexfn)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	254 if '00changelog' in indexfn:
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	255 raise util.Abort(_('shrinking the changelog '
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	256 'will corrupt your repository'))
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	257
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	258 ui.write(_('shrinking %s\n') % indexfn)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	259 prefix = os.path.basename(indexfn)[:-1]
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	260 (tmpfd, tmpindexfn) = tempfile.mkstemp(dir=os.path.dirname(indexfn),
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	261 prefix=prefix,
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	262 suffix='.i')
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	263 os.close(tmpfd)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	264
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	265 r1 = revlog.revlog(util.opener(os.getcwd(), audit=False), indexfn)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	266 r2 = revlog.revlog(util.opener(os.getcwd(), audit=False), tmpindexfn)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	267
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	268 datafn, tmpdatafn = r1.datafile, r2.datafile
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	269
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	270 oldindexfn = indexfn + '.old'
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	271 olddatafn = datafn + '.old'
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	272 if os.path.exists(oldindexfn) or os.path.exists(olddatafn):
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	273 raise util.Abort(_('one or both of\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	274 ' %s\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	275 ' %s\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	276 'exists from a previous run; please clean up '
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	277 'before running again') % (oldindexfn, olddatafn))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	278
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	279 # Don't use repo.transaction(), because then things get hairy with
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	280 # paths: some need to be relative to .hg, and some need to be
10216 843f6ee6d14b contrib: small documentation fixes in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10215 diff changeset	281 # absolute. Doing it this way keeps things simple: everything is an
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	282 # absolute path.
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	283 lock = repo.lock(wait=False)
10234 c8d6f339bbd7 shrink-revlog: make it work on windows (issue1976) Patrick Mezard <pmezard@gmail.com> parents: 10230 diff changeset	284 tr = transaction.transaction(ui.warn,
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	285 open,
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	286 repo.sjoin('journal'))
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	287
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	288 def ignoremissing(func):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	289 def f(args, *kw):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	290 try:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	291 return func(args, *kw)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	292 except OSError, inst:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	293 if inst.errno != errno.ENOENT:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	294 raise
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	295 return f
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	296
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	297 try:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	298 try:
10213 9e6848f352b0 contrib: use ui to write in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10009 diff changeset	299 order = toposort(ui, r1)
9e6848f352b0 contrib: use ui to write in shrink-revlog.py Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10009 diff changeset	300 writerevs(ui, r1, r2, order, tr)
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	301 report(ui, r1, r2)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	302 tr.close()
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	303 except:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	304 # Abort transaction first, so we truncate the files before
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	305 # deleting them.
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	306 tr.abort()
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	307 for fn in (tmpindexfn, tmpdatafn):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	308 ignoremissing(os.unlink)(fn)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	309 raise
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	310 if not opts.get('dry_run'):
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	311 # racy, both files cannot be renamed atomically
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	312 # copy files
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	313 util.os_link(indexfn, oldindexfn)
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	314 ignoremissing(util.os_link)(datafn, olddatafn)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	315 # rename
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	316 util.rename(tmpindexfn, indexfn)
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	317 try:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	318 util.rename(tmpdatafn, datafn)
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	319 except OSError, inst:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	320 if inst.errno != errno.ENOENT:
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	321 raise
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	322 ignoremissing(os.unlink)(datafn)
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	323 else:
10542 989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	324 for fn in (tmpindexfn, tmpdatafn):
989b2a5eaaba shrink: handle all combinations of inline/non-inline revlogs Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10509 diff changeset	325 ignoremissing(os.unlink)(fn)
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	326 finally:
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	327 lock.release()
f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	328
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	329 if not opts.get('dry_run'):
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	330 ui.write(_('note: old revlog saved in:\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	331 ' %s\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	332 ' %s\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	333 '(You can delete those files when you are satisfied that your\n'
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	334 'repository is still sane. '
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	335 'Running \'hg verify\' is strongly recommended.)\n')
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	336 % (oldindexfn, olddatafn))
9515 f7d85980261c Add script to rewrite revlog to workaround lack of parent deltas. Greg Ward <greg-hg@gerg.ca> parents: diff changeset	337
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	338 cmdtable = {
9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	339 'shrink': (shrink,
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	340 [('', 'revlog', '', _('index (.i) file of the revlog to shrink')),
cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	341 ('n', 'dry-run', None, _('do not shrink, simulate only')),
10622 bc81f126139f shrink-revlog: add --sort option for user-selectable toposort algorithm. Greg Ward <greg-hg@gerg.ca> parents: 10621 diff changeset	342 ('', 'sort', 'branchsort', 'name of sort algorithm to use'),
10241 4b2a086bee31 shrink-revlog: add --dry-run option Patrick Mezard <pmezard@gmail.com> parents: 10236 diff changeset	343 ],
10508 cc35ad583e66 shrink-revlog: add strings for translation / import _ before using it Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10496 diff changeset	344 _('hg shrink [--revlog PATH]'))
10215 9d79b8f58bea contrib: turn shrink-revlog.py into an extension Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 10214 diff changeset	345 }
10236 49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	346
49a8625b8cac shrink-revlog: help/doc tweaks Greg Ward <greg-hg@gerg.ca> parents: 10234 diff changeset	347 if __name__ == "__main__":
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10241 diff changeset	348 print "shrink-revlog.py is now an extension (see hg help extensions)"

Mercurial > public > mercurial-scm > hg

annotate contrib/shrink-revlog.py @ 10623:64e286c22f29