mercurial-scm/hg: hgext/convert/git.py annotate

annotate hgext/convert/git.py @ 21630:a204fd9b5ba9

convert: drastically speed up git conversions We would formerly exec git cat-file once for every commit, plus once for every tree and file we wnated to read. This switches to using git cat-file's batch mode, which is much, much, much faster. Using this new code, converting the git git repo to hg ran in 106 minutes on my machine. Using the stock mercurial, it required 1239 minutes. I believe this to be typical of the speedups we will see form this patch.

author	David Schleimer <dschleimer@fb.com>
date	Tue, 27 May 2014 21:12:24 -0700
parents	e8203629371b
children	cf599f8a2da8

rev	line source
8250 1b60efdb8bc5 convert: add copyright and license headers to back-ends Martin Geisler <mg@lazybytes.net> parents: 7875 diff changeset	1 # git.py - git support for the convert extension
1b60efdb8bc5 convert: add copyright and license headers to back-ends Martin Geisler <mg@lazybytes.net> parents: 7875 diff changeset	2 #
1b60efdb8bc5 convert: add copyright and license headers to back-ends Martin Geisler <mg@lazybytes.net> parents: 7875 diff changeset	3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
1b60efdb8bc5 convert: add copyright and license headers to back-ends Martin Geisler <mg@lazybytes.net> parents: 7875 diff changeset	4 #
1b60efdb8bc5 convert: add copyright and license headers to back-ends Martin Geisler <mg@lazybytes.net> parents: 7875 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 8456 diff changeset	6 # GNU General Public License version 2 or any later version.
3821 158fce02dc40 Teach convert-repo to deal with mixed charsets in git Matt Mackall <mpm@selenic.com> parents: 2657 diff changeset	7
4536 cc9b79216a76 Split convert extension into common and repository type modules Brendan Cully <brendan@kublai.com> parents: 4532 diff changeset	8 import os
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	9 import subprocess
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	10 from mercurial import util, config
12144 be9c4131a8f4 clone, patch, convert: use hex(nullid) instead of '0'40 Martin Geisler <mg@lazybytes.net>* parents: 11134 diff changeset	11 from mercurial.node import hex, nullid
10939 9f6731b03906 convert: mark strings for translation Martin Geisler <mg@lazybytes.net> parents: 10938 diff changeset	12 from mercurial.i18n import _
3938 0fab73b3f453 convert-repo: add some smarts Matt Mackall <mpm@selenic.com> parents: 3917 diff changeset	13
5497 f0a3918abd42 convert: fail if an external required tool is not found Patrick Mezard <pmezard@gmail.com> parents: 5404 diff changeset	14 from common import NoRepo, commit, converter_source, checktool
3954 9af4b853ed4d convert-repo: add CVS branch support Matt Mackall <mpm@selenic.com> parents: 3953 diff changeset	15
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	16 class submodule(object):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	17 def __init__(self, path, node, url):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	18 self.path = path
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	19 self.node = node
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	20 self.url = url
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	21
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	22 def hgsub(self):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	23 return "%s = [git]%s" % (self.path, self.url)
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	24
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	25 def hgsubstate(self):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	26 return "%s %s" % (self.node, self.path)
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	27
4448 af013ae3ca10 use documented convert-repo interface Daniel Holth <dholth@fastmail.fm> parents: 4447 diff changeset	28 class convert_git(converter_source):
5217 149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	29 # Windows does not support GIT_DIR= construct while other systems
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	30 # cannot remove environment variable. Just assume none have
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	31 # both issues.
14945 11aad09a6370 hgext: replace uses of hasattr with util.safehasattr Augie Fackler <durin42@gmail.com> parents: 14735 diff changeset	32 if util.safehasattr(os, 'unsetenv'):
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	33 def gitopen(self, s, err=None):
5217 149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	34 prevgitdir = os.environ.get('GIT_DIR')
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	35 os.environ['GIT_DIR'] = self.path
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	36 try:
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	37 if err == subprocess.PIPE:
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	38 (stdin, stdout, stderr) = util.popen3(s)
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	39 return stdout
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	40 elif err == subprocess.STDOUT:
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	41 return self.popen_with_stderr(s)
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	42 else:
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	43 return util.popen(s, 'rb')
5217 149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	44 finally:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	45 if prevgitdir is None:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	46 del os.environ['GIT_DIR']
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	47 else:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	48 os.environ['GIT_DIR'] = prevgitdir
21630 a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	49
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	50 def gitpipe(self, s):
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	51 prevgitdir = os.environ.get('GIT_DIR')
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	52 os.environ['GIT_DIR'] = self.path
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	53 try:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	54 return util.popen3(s)
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	55 finally:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	56 if prevgitdir is None:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	57 del os.environ['GIT_DIR']
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	58 else:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	59 os.environ['GIT_DIR'] = prevgitdir
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	60
5217 149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows. Patrick Mezard <pmezard@gmail.com> parents: 5216 diff changeset	61 else:
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	62 def gitopen(self, s, err=None):
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	63 if err == subprocess.PIPE:
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	64 (sin, so, se) = util.popen3('GIT_DIR=%s %s' % (self.path, s))
14177 fc004d16633f convert: fix error in git solaris code timeless <timeless@mozdev.org> parents: 13756 diff changeset	65 return so
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	66 elif err == subprocess.STDOUT:
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	67 return self.popen_with_stderr(s)
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	68 else:
14735 d297ee0b2d94 convert: fix git convert on solaris - it cannot remove environment variables Mads Kiilerich <mads@kiilerich.com> parents: 14177 diff changeset	69 return util.popen('GIT_DIR=%s %s' % (self.path, s), 'rb')
4767 2d0a823cbba5 convert: gitcmd wrapper for os.popen Brendan Cully <brendan@kublai.com> parents: 4760 diff changeset	70
21630 a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	71 def gitpipe(self, s):
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	72 return util.popen3('GIT_DIR=%s %s' % (self.path, s))
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	73
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	74 def popen_with_stderr(self, s):
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	75 p = subprocess.Popen(s, shell=True, bufsize=-1,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	76 close_fds=util.closefds,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	77 stdin=subprocess.PIPE,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	78 stdout=subprocess.PIPE,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	79 stderr=subprocess.STDOUT,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	80 universal_newlines=False,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	81 env=None)
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	82 return p.stdout
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	83
10986 610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	84 def gitread(self, s):
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	85 fh = self.gitopen(s)
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	86 data = fh.read()
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	87 return data, fh.close()
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	88
4760 07efcce17d28 convert: add -r argument specifying latest revision to convert Brendan Cully <brendan@kublai.com> parents: 4759 diff changeset	89 def __init__(self, ui, path, rev=None):
4807 15a3cbfc6568 convert: call superclass init from engine init functions Brendan Cully <brendan@kublai.com> parents: 4768 diff changeset	90 super(convert_git, self).__init__(ui, path, rev=rev)
15a3cbfc6568 convert: call superclass init from engine init functions Brendan Cully <brendan@kublai.com> parents: 4768 diff changeset	91
3938 0fab73b3f453 convert-repo: add some smarts Matt Mackall <mpm@selenic.com> parents: 3917 diff changeset	92 if os.path.isdir(path + "/.git"):
0fab73b3f453 convert-repo: add some smarts Matt Mackall <mpm@selenic.com> parents: 3917 diff changeset	93 path += "/.git"
4102 06d65498f73b convert-repo: use .git/objects/ rather than .git/HEAD to detect git repos Matt Mackall <mpm@selenic.com> parents: 4082 diff changeset	94 if not os.path.exists(path + "/objects"):
10939 9f6731b03906 convert: mark strings for translation Martin Geisler <mg@lazybytes.net> parents: 10938 diff changeset	95 raise NoRepo(_("%s does not look like a Git repository") % path)
5497 f0a3918abd42 convert: fail if an external required tool is not found Patrick Mezard <pmezard@gmail.com> parents: 5404 diff changeset	96
6837 e30c56f337b1 convert: use git executable only, with subcommands Dhruva Krishnamurthy <dhruvakm@gmail.com> parents: 6001 diff changeset	97 checktool('git', 'git')
5497 f0a3918abd42 convert: fail if an external required tool is not found Patrick Mezard <pmezard@gmail.com> parents: 5404 diff changeset	98
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	99 self.path = path
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	100 self.submodules = []
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	101
21630 a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	102 self.catfilepipe = self.gitpipe('git cat-file --batch')
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	103
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	104 def after(self):
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	105 for f in self.catfilepipe:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	106 f.close()
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	107
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	108 def getheads(self):
4768 f52bfe566583 convert: import all branches from git repositories Brendan Cully <brendan@kublai.com> parents: 4767 diff changeset	109 if not self.rev:
10986 610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	110 heads, ret = self.gitread('git rev-parse --branches --remotes')
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	111 heads = heads.splitlines()
4768 f52bfe566583 convert: import all branches from git repositories Brendan Cully <brendan@kublai.com> parents: 4767 diff changeset	112 else:
10986 610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	113 heads, ret = self.gitread("git rev-parse --verify %s" % self.rev)
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	114 heads = [heads[:-1]]
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	115 if ret:
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	116 raise util.Abort(_('cannot retrieve git heads'))
610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	117 return heads
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	118
692 695dd9a491da convert-repo: deal with packed git and other fixes mpm@selenic.com parents: 450 diff changeset	119 def catfile(self, rev, type):
12144 be9c4131a8f4 clone, patch, convert: use hex(nullid) instead of '0'40 Martin Geisler <mg@lazybytes.net>* parents: 11134 diff changeset	120 if rev == hex(nullid):
16687 e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException" Brodie Rao <brodie@sf.io> parents: 16683 diff changeset	121 raise IOError
21630 a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	122 self.catfilepipe[0].write(rev+'\n')
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	123 self.catfilepipe[0].flush()
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	124 info = self.catfilepipe[1].readline().split()
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	125 if info[1] != type:
10986 610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	126 raise util.Abort(_('cannot read %r object at %s') % (type, rev))
21630 a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	127 size = int(info[2])
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	128 data = self.catfilepipe[1].read(size)
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	129 if len(data) < size:
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	130 raise util.Abort(_('cannot read %r object at %s: %s') % (type, rev))
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	131 # read the trailing newline
a204fd9b5ba9 convert: drastically speed up git conversions David Schleimer <dschleimer@fb.com> parents: 20373 diff changeset	132 self.catfilepipe[1].read(1)
10986 610f047326b9 convert/git: check status when reading the whole output Patrick Mezard <pmezard@gmail.com> parents: 10985 diff changeset	133 return data
692 695dd9a491da convert-repo: deal with packed git and other fixes mpm@selenic.com parents: 450 diff changeset	134
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	135 def getfile(self, name, rev):
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	136 if name == '.hgsub':
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	137 data = '\n'.join([m.hgsub() for m in self.submoditer()])
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	138 mode = ''
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	139 elif name == '.hgsubstate':
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	140 data = '\n'.join([m.hgsubstate() for m in self.submoditer()])
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	141 mode = ''
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	142 else:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	143 data = self.catfile(rev, "blob")
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	144 mode = self.modecache[(name, rev)]
11134 33010ff1fd6f convert: merge sources getmode() into getfile() Patrick Mezard <pmezard@gmail.com> parents: 10987 diff changeset	145 return data, mode
3956 558f52943cd2 convert-repo: add CVS exec bit support Matt Mackall <mpm@selenic.com> parents: 3954 diff changeset	146
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	147 def submoditer(self):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	148 null = hex(nullid)
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	149 for m in sorted(self.submodules, key=lambda p: p.path):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	150 if m.node != null:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	151 yield m
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	152
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	153 def parsegitmodules(self, content):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	154 """Parse the formatted .gitmodules file, example file format:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	155 [submodule "sub"]\n
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	156 \tpath = sub\n
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	157 \turl = git://giturl\n
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	158 """
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	159 self.submodules = []
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	160 c = config.config()
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	161 # Each item in .gitmodules starts with \t that cant be parsed
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	162 c.parse('.gitmodules', content.replace('\t',''))
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	163 for sec in c.sections():
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	164 s = c[sec]
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	165 if 'url' in s and 'path' in s:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	166 self.submodules.append(submodule(s['path'], '', s['url']))
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	167
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	168 def retrievegitmodules(self, version):
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	169 modules, ret = self.gitread("git show %s:%s" % (version, '.gitmodules'))
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	170 if ret:
17930 7788b5e7d9ef convert: fix a too-long line nag Bryan O'Sullivan <bryano@fb.com> parents: 17929 diff changeset	171 raise util.Abort(_('cannot read submodules config file in %s') %
7788b5e7d9ef convert: fix a too-long line nag Bryan O'Sullivan <bryano@fb.com> parents: 17929 diff changeset	172 version)
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	173 self.parsegitmodules(modules)
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	174 for m in self.submodules:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	175 node, ret = self.gitread("git rev-parse %s:%s" % (version, m.path))
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	176 if ret:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	177 continue
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	178 m.node = node.strip()
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	179
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	180 def getchanges(self, version):
3956 558f52943cd2 convert-repo: add CVS exec bit support Matt Mackall <mpm@selenic.com> parents: 3954 diff changeset	181 self.modecache = {}
10985 7fab6ae3f688 convert/git: rename gitcmd() into gitopen() for readability Patrick Mezard <pmezard@gmail.com> parents: 10939 diff changeset	182 fh = self.gitopen("git diff-tree -z --root -m -r %s" % version)
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	183 changes = []
8456 e9e2a2c9b294 convert: use set instead of dict Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8271 diff changeset	184 seen = set()
7242 d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	185 entry = None
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	186 subexists = False
7242 d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	187 for l in fh.read().split('\x00'):
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	188 if not entry:
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	189 if not l.startswith(':'):
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	190 continue
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	191 entry = l
5335 88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5220 diff changeset	192 continue
7242 d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	193 f = l
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	194 if f not in seen:
8456 e9e2a2c9b294 convert: use set instead of dict Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8271 diff changeset	195 seen.add(f)
7242 d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	196 entry = entry.split()
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	197 h = entry[3]
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	198 p = (entry[1] == "100755")
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	199 s = (entry[1] == "120000")
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	200
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	201 if f == '.gitmodules':
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	202 subexists = True
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	203 changes.append(('.hgsub', ''))
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	204 elif entry[1] == '160000' or entry[0] == ':160000':
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	205 subexists = True
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	206 else:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	207 self.modecache[(f, h)] = (p and "x") or (s and "l") or ""
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	208 changes.append((f, h))
7242 d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360) Patrick Mezard <pmezard@gmail.com> parents: 7222 diff changeset	209 entry = None
10987 b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	210 if fh.close():
b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	211 raise util.Abort(_('cannot read changes in %s') % version)
17929 0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	212
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	213 if subexists:
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	214 self.retrievegitmodules(version)
0eed66327ad4 convert: add support for converting git submodule (issue3528) YaNan Xu <robot9@fb.com> parents: 16689 diff changeset	215 changes.append(('.hgsubstate', ''))
5121 ef338e34a906 convert: look up copies in getchanges instead of getcommit Brendan Cully <brendan@kublai.com> parents: 4873 diff changeset	216 return (changes, {})
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	217
c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	218 def getcommit(self, version):
692 695dd9a491da convert-repo: deal with packed git and other fixes mpm@selenic.com parents: 450 diff changeset	219 c = self.catfile(version, "commit") # read the commit hash
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	220 end = c.find("\n\n")
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	221 message = c[end + 2:]
4759 20ec5cc02f18 convert: ove recode method into converter_source Brendan Cully <brendan@kublai.com> parents: 4721 diff changeset	222 message = self.recode(message)
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	223 l = c[:end].splitlines()
c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	224 parents = []
8271 e3d3dad805f9 Add committer tag only when needed in git conversion Richard Quirk <richard.quirk@gmail.com> parents: 8250 diff changeset	225 author = committer = None
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	226 for e in l[1:]:
4532 c3a78a49d7f0 Some small cleanups for convert extension: Thomas Arendsen Hein <thomas@intevation.de> parents: 4521 diff changeset	227 n, v = e.split(" ", 1)
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	228 if n == "author":
c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	229 p = v.split()
1385 adb3de56635b convert-repo: Fix timezone handling Matt Mackall <mpm@selenic.com> parents: 1335 diff changeset	230 tm, tz = p[-2:]
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	231 author = " ".join(p[:-2])
c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	232 if author[0] == "<": author = author[1:-1]
4759 20ec5cc02f18 convert: ove recode method into converter_source Brendan Cully <brendan@kublai.com> parents: 4721 diff changeset	233 author = self.recode(author)
692 695dd9a491da convert-repo: deal with packed git and other fixes mpm@selenic.com parents: 450 diff changeset	234 if n == "committer":
431 dfc44f3f587c convert-repo fixups mpm@selenic.com parents: 316 diff changeset	235 p = v.split()
1385 adb3de56635b convert-repo: Fix timezone handling Matt Mackall <mpm@selenic.com> parents: 1335 diff changeset	236 tm, tz = p[-2:]
431 dfc44f3f587c convert-repo fixups mpm@selenic.com parents: 316 diff changeset	237 committer = " ".join(p[:-2])
dfc44f3f587c convert-repo fixups mpm@selenic.com parents: 316 diff changeset	238 if committer[0] == "<": committer = committer[1:-1]
4759 20ec5cc02f18 convert: ove recode method into converter_source Brendan Cully <brendan@kublai.com> parents: 4721 diff changeset	239 committer = self.recode(committer)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	240 if n == "parent":
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	241 parents.append(v)
1385 adb3de56635b convert-repo: Fix timezone handling Matt Mackall <mpm@selenic.com> parents: 1335 diff changeset	242
8271 e3d3dad805f9 Add committer tag only when needed in git conversion Richard Quirk <richard.quirk@gmail.com> parents: 8250 diff changeset	243 if committer and committer != author:
e3d3dad805f9 Add committer tag only when needed in git conversion Richard Quirk <richard.quirk@gmail.com> parents: 8250 diff changeset	244 message += "\ncommitter: %s\n" % committer
1385 adb3de56635b convert-repo: Fix timezone handling Matt Mackall <mpm@selenic.com> parents: 1335 diff changeset	245 tzs, tzh, tzm = tz[-5:-4] + "1", tz[-4:-2], tz[-2:]
2093 5cc414722587 convert-repo: fix reversed time zone offset Vadim Gelfer <vadim.gelfer@gmail.com> parents: 1715 diff changeset	246 tz = -int(tzs) * (int(tzh) * 3600 + int(tzm))
1385 adb3de56635b convert-repo: Fix timezone handling Matt Mackall <mpm@selenic.com> parents: 1335 diff changeset	247 date = tm + " " + str(tz)
3954 9af4b853ed4d convert-repo: add CVS branch support Matt Mackall <mpm@selenic.com> parents: 3953 diff changeset	248
4873 28b23b9073a8 convert: record the source revision in the changelog Brendan Cully <brendan@kublai.com> parents: 4810 diff changeset	249 c = commit(parents=parents, date=date, author=author, desc=message,
28b23b9073a8 convert: record the source revision in the changelog Brendan Cully <brendan@kublai.com> parents: 4810 diff changeset	250 rev=version)
3954 9af4b853ed4d convert-repo: add CVS branch support Matt Mackall <mpm@selenic.com> parents: 3953 diff changeset	251 return c
316 c48d069163d6 Add new convert-repo script mpm@selenic.com parents: diff changeset	252
694 51eb248d3348 Teach convert-repo about tags mpm@selenic.com parents: 692 diff changeset	253 def gettags(self):
51eb248d3348 Teach convert-repo about tags mpm@selenic.com parents: 692 diff changeset	254 tags = {}
16259 589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	255 alltags = {}
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	256 fh = self.gitopen('git ls-remote --tags "%s"' % self.path,
dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	257 err=subprocess.STDOUT)
4062 516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	258 prefix = 'refs/tags/'
16259 589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	259
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	260 # Build complete list of tags, both annotated and bare ones
4062 516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	261 for line in fh:
516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	262 line = line.strip()
18572 5fe58f9332a4 git convert: some versions of git use fatal: instead of error: Augie Fackler <raf@durin42.com> parents: 18570 diff changeset	263 if line.startswith("error:") or line.startswith("fatal:"):
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	264 raise util.Abort(_('cannot read tags from %s') % self.path)
4062 516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	265 node, tag = line.split(None, 1)
516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	266 if not tag.startswith(prefix):
516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	267 continue
16259 589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	268 alltags[tag[len(prefix):]] = node
10987 b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	269 if fh.close():
b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	270 raise util.Abort(_('cannot read tags from %s') % self.path)
4062 516f883e3d79 convert-repo: handle packed git tags Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4047 diff changeset	271
16259 589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	272 # Filter out tag objects for annotated tag refs
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	273 for tag in alltags:
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	274 if tag.endswith('^{}'):
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	275 tags[tag[:-3]] = alltags[tag]
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	276 else:
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	277 if tag + '^{}' in alltags:
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	278 continue
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	279 else:
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	280 tags[tag] = alltags[tag]
589aab2ca716 convert: support non annotated tags in git backend Edouard Gomez <ed.gomez@free.fr> parents: 14945 diff changeset	281
694 51eb248d3348 Teach convert-repo about tags mpm@selenic.com parents: 692 diff changeset	282 return tags
5380 a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	283
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	284 def getchangedfiles(self, version, i):
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	285 changes = []
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	286 if i is None:
10985 7fab6ae3f688 convert/git: rename gitcmd() into gitopen() for readability Patrick Mezard <pmezard@gmail.com> parents: 10939 diff changeset	287 fh = self.gitopen("git diff-tree --root -m -r %s" % version)
5380 a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	288 for l in fh:
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	289 if "\t" not in l:
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	290 continue
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	291 m, f = l[:-1].split("\t")
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	292 changes.append(f)
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	293 else:
16683 525fdb738975 cleanup: eradicate long lines Brodie Rao <brodie@sf.io> parents: 16295 diff changeset	294 fh = self.gitopen('git diff-tree --name-only --root -r %s '
525fdb738975 cleanup: eradicate long lines Brodie Rao <brodie@sf.io> parents: 16295 diff changeset	295 '"%s^%s" --' % (version, version, i + 1))
5380 a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	296 changes = [f.rstrip('\n') for f in fh]
10987 b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	297 if fh.close():
b3af02b1f19f convert/git: check status when reading output stream Patrick Mezard <pmezard@gmail.com> parents: 10986 diff changeset	298 raise util.Abort(_('cannot read changes in %s') % version)
5380 a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	299
a5a7f7fd5554 convert_git: add --filemap support Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 5336 diff changeset	300 return changes
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	301
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	302 def getbookmarks(self):
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	303 bookmarks = {}
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	304
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	305 # Interesting references in git are prefixed
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	306 prefix = 'refs/heads/'
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	307 prefixlen = len(prefix)
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	308
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	309 # factor two commands
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	310 gitcmd = { 'remote/': 'git ls-remote --heads origin',
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	311 '': 'git show-ref'}
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	312
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	313 # Origin heads
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	314 for reftype in gitcmd:
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	315 try:
18570 dcf2d6fdf630 convert/git: catch errors from modern git-ls-remote (issue3428) Ross Lagerwall <rosslagerwall@gmail.com> parents: 17930 diff changeset	316 fh = self.gitopen(gitcmd[reftype], err=subprocess.PIPE)
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	317 for line in fh:
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	318 line = line.strip()
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	319 rev, name = line.split(None, 1)
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	320 if not name.startswith(prefix):
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	321 continue
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	322 name = '%s%s' % (reftype, name[prefixlen:])
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	323 bookmarks[name] = rev
16689 f366d4c2ff34 cleanup: replace naked excepts with except Exception: ... Brodie Rao <brodie@sf.io> parents: 16687 diff changeset	324 except Exception:
13756 6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	325 pass
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	326
6b7077df4aa5 convert: add bookmarks reading support to git backend Edouard Gomez <ed.gomez@free.fr> parents: 12144 diff changeset	327 return bookmarks
19121 478a04605ce1 splicemap: improve error handling when source is git (issue2084) Ben Goswami <bengoswami@fb.com> parents: 18572 diff changeset	328
20373 e8203629371b convert: add mapname parameter to checkrevformat Sean Farley <sean.michael.farley@gmail.com> parents: 19121 diff changeset	329 def checkrevformat(self, revstr, mapname='splicemap'):
19121 478a04605ce1 splicemap: improve error handling when source is git (issue2084) Ben Goswami <bengoswami@fb.com> parents: 18572 diff changeset	330 """ git revision string is a 40 byte hex """
20373 e8203629371b convert: add mapname parameter to checkrevformat Sean Farley <sean.michael.farley@gmail.com> parents: 19121 diff changeset	331 self.checkhexformat(revstr, mapname)
19121 478a04605ce1 splicemap: improve error handling when source is git (issue2084) Ben Goswami <bengoswami@fb.com> parents: 18572 diff changeset	332

Mercurial > public > mercurial-scm > hg

annotate hgext/convert/git.py @ 21630:a204fd9b5ba9