mercurial-scm/hg: mercurial/changegroup.py annotate

annotate mercurial/changegroup.py @ 23381:cc0ff93d0c0c stable

changegroup: fix file linkrevs during reorders (issue4462) Previously, if reorder was true during the creation of a changegroup bundle, it was possible that the manifest and filelogs would be reordered such that the resulting bundle filelog had a linkrev that pointed to a commit that was not the earliest instance of the filelog revision. For example: With commits: 0<-1<---3<-4 \ / --2<--- if 2 and 3 added the same version of a file, if the manifests of 2 and 3 have their order reversed, but the changelog did not, it could produce a filelog with linkrevs 0<-3 instead of 0<-2, which meant if commit 3 was stripped, it would delete that file data from the repository and commit 2 would be corrupt (as would any future pulls that tried to build upon that version of the file). The fix is to make the linkrev fixup smarter. Previously it considered the first manifest that added a file to be the first commit that added that file, which is not true. Now, for every file revision we add to the bundle we make sure we attach it to the earliest applicable linkrev.

author	Durham Goode <durham@fb.com>
date	Thu, 20 Nov 2014 16:30:57 -0800
parents	5dcaed20b27c
children	a81c76106d90

rev	line source
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9437 diff changeset	6 # GNU General Public License version 2 or any later version.
3877 abaee83ce0a6 Replace demandload with new demandimport Matt Mackall <mpm@selenic.com> parents: 3859 diff changeset	7
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	8 import weakref
3891 6b4127c7d52a Simplify i18n imports Matt Mackall <mpm@selenic.com> parents: 3877 diff changeset	9 from i18n import _
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	10 from node import nullrev, nullid, hex, short
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	11 import mdiff, util, dagutil
8312 b87a50b7125c separate import lines from mercurial and general python modules Simon Heimberg <simohe@besonet.ch> parents: 8226 diff changeset	12 import struct, os, bz2, zlib, tempfile
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	13 import discovery, error, phases, branchmap
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	14
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	15 _CHANGEGROUPV1_DELTA_HEADER = "20s20s20s20s"
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	16
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	17 def readexactly(stream, n):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	18 '''read n bytes from stream.read and abort if less was available'''
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	19 s = stream.read(n)
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	20 if len(s) < n:
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	21 raise util.Abort(_("stream ended unexpectedly"
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	22 " (got %d bytes, expected %d)")
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	23 % (len(s), n))
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	24 return s
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	25
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	26 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	27 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	28 d = readexactly(stream, 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	29 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	30 if l <= 4:
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	31 if l:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	32 raise util.Abort(_("invalid chunk length %d") % l)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	33 return ""
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	34 return readexactly(stream, l - 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	35
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	36 def chunkheader(length):
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	37 """return a changegroup chunk header (string)"""
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	38 return struct.pack(">l", length + 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	39
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	40 def closechunk():
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	41 """return a changegroup chunk header (string) for a zero-length chunk"""
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	42 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	43
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	44 class nocompress(object):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	45 def compress(self, x):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	46 return x
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	47 def flush(self):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	48 return ""
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	49
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	50 bundletypes = {
14060 aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	51 "": ("", nocompress), # only when using unbundle on ssh and old http servers
aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	52 # since the unification ssh accepts a header but there
aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	53 # is no capability signaling it.
3704 9c1737a3e254 fix writebundle for bz2 bundles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3662 diff changeset	54 "HG10UN": ("HG10UN", nocompress),
3762 b9d3e12da485 changegroup.py: delay the loading of the bz2 and zlib modules Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3706 diff changeset	55 "HG10BZ": ("HG10", lambda: bz2.BZ2Compressor()),
b9d3e12da485 changegroup.py: delay the loading of the bz2 and zlib modules Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3706 diff changeset	56 "HG10GZ": ("HG10GZ", lambda: zlib.compressobj()),
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	57 }
f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	58
9087 f48454a279b9 typos: "it's" -> "its" Martin Geisler <mg@lazybytes.net> parents: 8312 diff changeset	59 # hgweb uses this list to communicate its preferred type
6152 c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	60 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	61
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	62 def writebundle(cg, filename, bundletype, vfs=None):
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	63 """Write a bundle file and return its filename.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	64
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	65 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	66 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	67 bz2 compression can be turned off.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	68 The bundle file will be deleted in case of errors.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	69 """
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	70
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	71 fh = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	72 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	73 try:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	74 if filename:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	75 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	76 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	77 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	78 fh = open(filename, "wb")
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	79 else:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	80 fd, filename = tempfile.mkstemp(prefix="hg-bundle-", suffix=".hg")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	81 fh = os.fdopen(fd, "wb")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	82 cleanup = filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	83
3706 0d810798acb1 Use 'bundletype' instead of 'type' to not shadow built-in function. Thomas Arendsen Hein <thomas@intevation.de> parents: 3705 diff changeset	84 header, compressor = bundletypes[bundletype]
3704 9c1737a3e254 fix writebundle for bz2 bundles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3662 diff changeset	85 fh.write(header)
9c1737a3e254 fix writebundle for bz2 bundles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3662 diff changeset	86 z = compressor()
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	87
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	88 # parse the changegroup data, otherwise we will block
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	89 # in case of sshrepo because we don't know the end of the stream
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	90
12335 e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	91 # an empty chunkgroup is the end of the changegroup
e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	92 # a changegroup has at least 2 chunkgroups (changelog and manifest).
e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	93 # after that, an empty chunkgroup is the end of the changegroup
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	94 for chunk in cg.getchunks():
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	95 fh.write(z.compress(chunk))
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	96 fh.write(z.flush())
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	97 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	98 return filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	99 finally:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	100 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	101 fh.close()
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	102 if cleanup is not None:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	103 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	104 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	105 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	106 os.unlink(cleanup)
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	107
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	108 def decompressor(fh, alg):
270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	109 if alg == 'UN':
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	110 return fh
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	111 elif alg == 'GZ':
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	112 def generator(f):
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	113 zd = zlib.decompressobj()
16557 9dba55369cd8 changegroup: decompress GZ algorithm in larger chunks for better performance Michael Tj?rnemark <michael@tjornemark.dk> parents: 14494 diff changeset	114 for chunk in util.filechunkiter(f):
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	115 yield zd.decompress(chunk)
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	116 elif alg == 'BZ':
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	117 def generator(f):
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	118 zd = bz2.BZ2Decompressor()
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	119 zd.decompress("BZ")
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	120 for chunk in util.filechunkiter(f, 4096):
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	121 yield zd.decompress(chunk)
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	122 else:
270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	123 raise util.Abort("unknown bundle compression '%s'" % alg)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	124 return util.chunkbuffer(generator(fh))
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	125
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	126 class cg1unpacker(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	127 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	128 deltaheadersize = struct.calcsize(deltaheader)
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	129 def __init__(self, fh, alg):
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	130 self._stream = decompressor(fh, alg)
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	131 self._type = alg
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	132 self.callback = None
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	133 def compressed(self):
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	134 return self._type != 'UN'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	135 def read(self, l):
bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	136 return self._stream.read(l)
12330 e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	137 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	138 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	139 def tell(self):
12332 680fe77ab5b8 bundlerepo: use bundle objects everywhere Matt Mackall <mpm@selenic.com> parents: 12330 diff changeset	140 return self._stream.tell()
12347 6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	141 def close(self):
6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	142 return self._stream.close()
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	143
50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	144 def chunklength(self):
13459 acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34 Jim Hague <jim.hague@acm.org> parents: 13458 diff changeset	145 d = readexactly(self._stream, 4)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	146 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	147 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	148 if l:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	149 raise util.Abort(_("invalid chunk length %d") % l)
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	150 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	151 if self.callback:
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	152 self.callback()
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	153 return l - 4
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	154
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	155 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	156 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	157 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	158
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	159 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	160 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	161 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	162
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	163 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	164 """return the header of the filelogs chunk, v10 only has the filename"""
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	165 l = self.chunklength()
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	166 if not l:
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	167 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	168 fname = readexactly(self._stream, l)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	169 return {'filename': fname}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	170
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	171 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	172 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	173 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	174 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	175 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	176 deltabase = prevnode
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	177 return node, p1, p2, deltabase, cs
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	178
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	179 def deltachunk(self, prevnode):
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	180 l = self.chunklength()
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	181 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	182 return {}
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	183 headerdata = readexactly(self._stream, self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	184 header = struct.unpack(self.deltaheader, headerdata)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	185 delta = readexactly(self._stream, l - self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	186 node, p1, p2, deltabase, cs = self._deltaheader(header, prevnode)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	187 return {'node': node, 'p1': p1, 'p2': p2, 'cs': cs,
f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	188 'deltabase': deltabase, 'delta': delta}
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	189
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	190 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	191 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	192
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	193 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	194 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	195 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	196 """
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	197 # an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	198 # a changegroup has at least 2 chunkgroups (changelog and manifest).
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	199 # after that, an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	200 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	201 count = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	202 while not empty or count <= 2:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	203 empty = True
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	204 count += 1
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	205 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	206 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	207 if not chunk:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	208 break
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	209 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	210 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	211 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	212 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	213 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	214 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	215 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	216 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	217
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	218 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	219 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	220 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	221 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	222 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	223 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	224 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	225 if len(d) < n:
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	226 d += readexactly(self._fh, n - len(d))
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	227 return d
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	228 return readexactly(self._fh, n)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	229
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	230 class cg1packer(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	231 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	232 def __init__(self, repo, bundlecaps=None):
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	233 """Given a source repo, construct a bundler.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	234
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	235 bundlecaps is optional and can be used to specify the set of
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	236 capabilities which can be used to build the bundle.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	237 """
19201 309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	238 # Set of capabilities we can use to build the bundle.
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	239 if bundlecaps is None:
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	240 bundlecaps = set()
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	241 self._bundlecaps = bundlecaps
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	242 self._changelog = repo.changelog
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	243 self._manifest = repo.manifest
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	244 reorder = repo.ui.config('bundle', 'reorder', 'auto')
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	245 if reorder == 'auto':
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	246 reorder = None
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	247 else:
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	248 reorder = util.parsebool(reorder)
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	249 self._repo = repo
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	250 self._reorder = reorder
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	251 self._progress = repo.ui.progress
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	252 def close(self):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	253 return closechunk()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	254
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	255 def fileheader(self, fname):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	256 return chunkheader(len(fname)) + fname
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	257
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	258 def group(self, nodelist, revlog, lookup, units=None, reorder=None):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	259 """Calculate a delta group, yielding a sequence of changegroup chunks
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	260 (strings).
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	261
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	262 Given a list of changeset revs, return a set of deltas and
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	263 metadata corresponding to nodes. The first delta is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	264 first parent(nodelist[0]) -> nodelist[0], the receiver is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	265 guaranteed to have this parent as it has all history before
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	266 these changesets. In the case firstparent is nullrev the
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	267 changegroup starts with a full revision.
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	268
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	269 If units is not None, progress detail will be generated, units specifies
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	270 the type of revlog that is touched (changelog, manifest, etc.).
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	271 """
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	272 # if we don't have any revisions touched by these changesets, bail
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	273 if len(nodelist) == 0:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	274 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	275 return
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	276
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	277 # for generaldelta revlogs, we linearize the revs; this will both be
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	278 # much quicker and generate a much smaller bundle
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	279 if (revlog._generaldelta and reorder is not False) or reorder:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	280 dag = dagutil.revlogdag(revlog)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	281 revs = set(revlog.rev(n) for n in nodelist)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	282 revs = dag.linearize(revs)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	283 else:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	284 revs = sorted([revlog.rev(n) for n in nodelist])
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	285
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	286 # add the parent of the first rev
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	287 p = revlog.parentrevs(revs[0])[0]
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	288 revs.insert(0, p)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	289
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	290 # build deltas
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	291 total = len(revs) - 1
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	292 msgbundling = _('bundling')
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	293 for r in xrange(len(revs) - 1):
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	294 if units is not None:
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	295 self._progress(msgbundling, r + 1, unit=units, total=total)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	296 prev, curr = revs[r], revs[r + 1]
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	297 linknode = lookup(revlog.node(curr))
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	298 for c in self.revchunk(revlog, curr, prev, linknode):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	299 yield c
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	300
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	301 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	302
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	303 # filter any nodes that claim to be part of the known set
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	304 def prune(self, revlog, missing, commonrevs, source):
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	305 rr, rl = revlog.rev, revlog.linkrev
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	306 return [n for n in missing if rl(rr(n)) not in commonrevs]
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	307
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	308 def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	309 '''yield a sequence of changegroup chunks (strings)'''
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	310 repo = self._repo
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	311 cl = self._changelog
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	312 mf = self._manifest
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	313 reorder = self._reorder
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	314 progress = self._progress
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	315
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	316 # for progress output
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	317 msgbundling = _('bundling')
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	318
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	319 clrevorder = {}
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	320 mfs = {} # needed manifests
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	321 fnodes = {} # needed file nodes
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	322 changedfiles = set()
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	323
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	324 # Callback for the changelog, used to collect changed files and manifest
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	325 # nodes.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	326 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	327 def lookupcl(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	328 c = cl.read(x)
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	329 clrevorder[x] = len(clrevorder)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	330 changedfiles.update(c[3])
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	331 # record the first changeset introducing this manifest version
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	332 mfs.setdefault(c[0], x)
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	333 return x
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	334
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	335 for chunk in self.group(clnodes, cl, lookupcl, units=_('changesets'),
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	336 reorder=reorder):
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	337 yield chunk
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	338 progress(msgbundling, None)
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	339
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	340 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	341 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	342 # Returns the linkrev node (collected in lookupcl).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	343 def lookupmf(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	344 clnode = mfs[x]
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	345 if not fastpathlinkrev or reorder:
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	346 mdata = mf.readfast(x)
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	347 for f, n in mdata.iteritems():
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	348 if f in changedfiles:
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	349 # record the first changeset introducing this filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	350 # version
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	351 fclnodes = fnodes.setdefault(f, {})
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	352 fclnode = fclnodes.setdefault(n, clnode)
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	353 if clrevorder[clnode] < clrevorder[fclnode]:
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	354 fclnodes[n] = clnode
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	355 return clnode
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	356
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	357 mfnodes = self.prune(mf, mfs, commonrevs, source)
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	358 for chunk in self.group(mfnodes, mf, lookupmf, units=_('manifests'),
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	359 reorder=reorder):
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	360 yield chunk
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	361 progress(msgbundling, None)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	362
6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	363 mfs.clear()
19708 fd4f612f7cb6 bundle: fix performance regression when bundling file changes (issue4031) Antoine Pitrou <solipsis@pitrou.net> parents: 19334 diff changeset	364 needed = set(cl.rev(x) for x in clnodes)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	365
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	366 def linknodes(filerevlog, fname):
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	367 if fastpathlinkrev and not reorder:
20936 bfb40168391c changegroup: remove unused variable caught by pyflakes Sean Farley <sean.michael.farley@gmail.com> parents: 19708 diff changeset	368 llr = filerevlog.linkrev
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	369 def genfilenodes():
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	370 for r in filerevlog:
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	371 linkrev = llr(r)
19325 ec896f9e8894 changegroup: fix fastpath during commit Matt Mackall <mpm@selenic.com> parents: 19293 diff changeset	372 if linkrev in needed:
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	373 yield filerevlog.node(r), cl.node(linkrev)
23225 bdf7b1ea1dae changegroup: don't store unused value on fnodes (issue4443) Gregory Szorc <gregory.szorc@gmail.com> parents: 23224 diff changeset	374 return dict(genfilenodes())
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	375 return fnodes.get(fname, {})
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	376
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	377 for chunk in self.generatefiles(changedfiles, linknodes, commonrevs,
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	378 source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	379 yield chunk
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	380
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	381 yield self.close()
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	382 progress(msgbundling, None)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	383
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	384 if clnodes:
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	385 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	386
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	387 def generatefiles(self, changedfiles, linknodes, commonrevs, source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	388 repo = self._repo
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	389 progress = self._progress
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	390 reorder = self._reorder
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	391 msgbundling = _('bundling')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	392
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	393 total = len(changedfiles)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	394 # for progress output
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	395 msgfiles = _('files')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	396 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	397 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	398 if not filerevlog:
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	399 raise util.Abort(_("empty or missing revlog for %s") % fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	400
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	401 linkrevnodes = linknodes(filerevlog, fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	402 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	403 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	404 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	405 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	406
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	407 filenodes = self.prune(filerevlog, linkrevnodes, commonrevs, source)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	408 if filenodes:
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	409 progress(msgbundling, i + 1, item=fname, unit=msgfiles,
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	410 total=total)
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	411 yield self.fileheader(fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	412 for chunk in self.group(filenodes, filerevlog, lookupfilelog,
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	413 reorder=reorder):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	414 yield chunk
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	415
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	416 def revchunk(self, revlog, rev, prev, linknode):
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	417 node = revlog.node(rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	418 p1, p2 = revlog.parentrevs(rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	419 base = prev
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	420
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	421 prefix = ''
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	422 if base == nullrev:
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	423 delta = revlog.revision(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	424 prefix = mdiff.trivialdiffheader(len(delta))
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	425 else:
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	426 delta = revlog.revdiff(base, rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	427 p1n, p2n = revlog.parents(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	428 basenode = revlog.node(base)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	429 meta = self.builddeltaheader(node, p1n, p2n, basenode, linknode)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	430 meta += prefix
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	431 l = len(meta) + len(delta)
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	432 yield chunkheader(l)
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	433 yield meta
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	434 yield delta
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	435 def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	436 # do nothing with basenode, it is implicitly the previous one in HG10
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	437 return struct.pack(self.deltaheader, node, p1n, p2n, linknode)
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	438
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	439 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	440 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	441 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	442 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	443 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	444 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	445 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	446
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	447 def getsubset(repo, outgoing, bundler, source, fastpath=False):
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	448 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	449 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	450 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	451 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	452 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	453 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	454 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	455 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	456 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	457 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	458
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	459 repo.hook('preoutgoing', throw=True, source=source)
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	460 _changegroupinfo(repo, csets, source)
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	461 gengroup = bundler.generate(commonrevs, csets, fastpathlinkrev, source)
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	462 return cg1unpacker(util.chunkbuffer(gengroup), 'UN')
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	463
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	464 def changegroupsubset(repo, roots, heads, source):
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	465 """Compute a changegroup consisting of all the nodes that are
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	466 descendants of any of the roots and ancestors of any of the heads.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	467 Return a chunkbuffer object whose read() method will return
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	468 successive changegroup chunks.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	469
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	470 It is fairly complex as determining which filenodes and which
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	471 manifest nodes need to be included for the changeset to be complete
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	472 is non-trivial.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	473
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	474 Another wrinkle is doing the reverse, figuring out which changeset in
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	475 the changegroup a particular filenode or manifestnode belongs to.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	476 """
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	477 cl = repo.changelog
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	478 if not roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	479 roots = [nullid]
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	480 # TODO: remove call to nodesbetween.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	481 csets, roots, heads = cl.nodesbetween(roots, heads)
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	482 discbases = []
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	483 for n in roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	484 discbases.extend([p for p in cl.parents(n) if p != nullid])
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	485 outgoing = discovery.outgoing(cl, discbases, heads)
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	486 bundler = cg1packer(repo)
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	487 return getsubset(repo, outgoing, bundler, source)
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	488
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	489 def getlocalchangegroup(repo, source, outgoing, bundlecaps=None):
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	490 """Like getbundle, but taking a discovery.outgoing as an argument.
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	491
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	492 This is only implemented for local repos and reuses potentially
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	493 precomputed sets in outgoing."""
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	494 if not outgoing.missing:
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	495 return None
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	496 bundler = cg1packer(repo, bundlecaps)
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	497 return getsubset(repo, outgoing, bundler, source)
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	498
21260 aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	499 def _computeoutgoing(repo, heads, common):
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	500 """Computes which revs are outgoing given a set of common
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	501 and a set of heads.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	502
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	503 This is a separate function so extensions can have access to
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	504 the logic.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	505
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	506 Returns a discovery.outgoing object.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	507 """
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	508 cl = repo.changelog
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	509 if common:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	510 hasnode = cl.hasnode
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	511 common = [n for n in common if hasnode(n)]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	512 else:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	513 common = [nullid]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	514 if not heads:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	515 heads = cl.heads()
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	516 return discovery.outgoing(cl, common, heads)
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	517
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	518 def getchangegroup(repo, source, heads=None, common=None, bundlecaps=None):
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	519 """Like changegroupsubset, but returns the set difference between the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	520 ancestors of heads and the ancestors common.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	521
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	522 If heads is None, use the local heads. If common is None, use [nullid].
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	523
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	524 The nodes in common might not all be known locally due to the way the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	525 current discovery protocol works.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	526 """
21260 aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	527 outgoing = _computeoutgoing(repo, heads, common)
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	528 return getlocalchangegroup(repo, source, outgoing, bundlecaps=bundlecaps)
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	529
20931 de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	530 def changegroup(repo, basenodes, source):
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	531 # to avoid a race we use changegroupsubset() (issue1320)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	532 return changegroupsubset(repo, basenodes, repo.heads(), source)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	533
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	534 def addchangegroupfiles(repo, source, revmap, trp, pr, needfiles):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	535 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	536 files = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	537 while True:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	538 chunkdata = source.filelogheader()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	539 if not chunkdata:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	540 break
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	541 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	542 repo.ui.debug("adding %s revisions\n" % f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	543 pr()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	544 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	545 o = len(fl)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	546 if not fl.addgroup(source, revmap, trp):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	547 raise util.Abort(_("received file revlog group is empty"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	548 revisions += len(fl) - o
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	549 files += 1
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	550 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	551 needs = needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	552 for new in xrange(o, len(fl)):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	553 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	554 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	555 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	556 else:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	557 raise util.Abort(
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	558 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	559 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	560 del needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	561 repo.ui.progress(_('files'), None)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	562
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	563 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	564 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	565 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	566 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	567 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	568 except error.LookupError:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	569 raise util.Abort(
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	570 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	571 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	572
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	573 return revisions, files
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	574
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	575 def addchangegroup(repo, source, srctype, url, emptyok=False,
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	576 targetphase=phases.draft):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	577 """Add the changegroup returned by source.read() to this repo.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	578 srctype is a string like 'push', 'pull', or 'unbundle'. url is
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	579 the URL of the repo where this changegroup is coming from.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	580
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	581 Return an integer summarizing the change to this repo:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	582 - nothing changed or no source: 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	583 - more heads than before: 1+added heads (2..n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	584 - fewer heads than before: -1-removed heads (-2..-n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	585 - number of heads stays the same: 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	586 """
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	587 repo = repo.unfiltered()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	588 def csmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	589 repo.ui.debug("add changeset %s\n" % short(x))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	590 return len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	591
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	592 def revmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	593 return cl.rev(x)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	594
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	595 if not source:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	596 return 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	597
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	598 changesets = files = revisions = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	599 efiles = set()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	600
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	601 # write changelog data to temp files so concurrent readers will not see
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	602 # inconsistent view
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	603 cl = repo.changelog
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	604 cl.delayupdate()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	605 oldheads = cl.heads()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	606
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	607 tr = repo.transaction("\n".join([srctype, util.hidepassword(url)]))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	608 # The transaction could have been created before and already carries source
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	609 # information. In this case we use the top level data. We overwrite the
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	610 # argument because we need to use the top level value (if they exist) in
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	611 # this function.
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	612 srctype = tr.hookargs.setdefault('source', srctype)
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	613 url = tr.hookargs.setdefault('url', url)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	614 try:
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	615 repo.hook('prechangegroup', throw=True, **tr.hookargs)
22969 805c18b27f48 addchangegroup: call `prechangegroup` hook after transaction retrieval Pierre-Yves David <pierre-yves.david@fb.com> parents: 22968 diff changeset	616
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	617 trp = weakref.proxy(tr)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	618 # pull off the changeset group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	619 repo.ui.status(_("adding changesets\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	620 clstart = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	621 class prog(object):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	622 step = _('changesets')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	623 count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	624 ui = repo.ui
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	625 total = None
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	626 def __call__(repo):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	627 repo.ui.progress(repo.step, repo.count, unit=_('chunks'),
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	628 total=repo.total)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	629 repo.count += 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	630 pr = prog()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	631 source.callback = pr
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	632
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	633 source.changelogheader()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	634 srccontent = cl.addgroup(source, csmap, trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	635 if not (srccontent or emptyok):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	636 raise util.Abort(_("received changelog group is empty"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	637 clend = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	638 changesets = clend - clstart
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	639 for c in xrange(clstart, clend):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	640 efiles.update(repo[c].files())
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	641 efiles = len(efiles)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	642 repo.ui.progress(_('changesets'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	643
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	644 # pull off the manifest group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	645 repo.ui.status(_("adding manifests\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	646 pr.step = _('manifests')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	647 pr.count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	648 pr.total = changesets # manifests <= changesets
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	649 # no need to check for empty manifest group here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	650 # if the result of the merge of 1 and 2 is the same in 3 and 4,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	651 # no new manifest will be created and the manifest group will
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	652 # be empty during the pull
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	653 source.manifestheader()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	654 repo.manifest.addgroup(source, revmap, trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	655 repo.ui.progress(_('manifests'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	656
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	657 needfiles = {}
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	658 if repo.ui.configbool('server', 'validate', default=False):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	659 # validate incoming csets have their manifests
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	660 for cset in xrange(clstart, clend):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	661 mfest = repo.changelog.read(repo.changelog.node(cset))[0]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	662 mfest = repo.manifest.readdelta(mfest)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	663 # store file nodes we must see
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	664 for f, n in mfest.iteritems():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	665 needfiles.setdefault(f, set()).add(n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	666
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	667 # process the files
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	668 repo.ui.status(_("adding file changes\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	669 pr.step = _('files')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	670 pr.count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	671 pr.total = efiles
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	672 source.callback = None
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	673
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	674 newrevs, newfiles = addchangegroupfiles(repo, source, revmap, trp, pr,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	675 needfiles)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	676 revisions += newrevs
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	677 files += newfiles
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	678
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	679 dh = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	680 if oldheads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	681 heads = cl.heads()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	682 dh = len(heads) - len(oldheads)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	683 for h in heads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	684 if h not in oldheads and repo[h].closesbranch():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	685 dh -= 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	686 htext = ""
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	687 if dh:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	688 htext = _(" (%+d heads)") % dh
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	689
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	690 repo.ui.status(_("added %d changesets"
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	691 " with %d changes to %d files%s\n")
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	692 % (changesets, revisions, files, htext))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	693 repo.invalidatevolatilesets()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	694
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	695 if changesets > 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	696 p = lambda: cl.writepending() and repo.root or ""
21151 240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	697 if 'node' not in tr.hookargs:
240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	698 tr.hookargs['node'] = hex(cl.node(clstart))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	699 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	700 else:
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	701 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	702 hookargs['node'] = hex(cl.node(clstart))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	703 repo.hook('pretxnchangegroup', throw=True, pending=p, **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	704
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	705 added = [cl.node(r) for r in xrange(clstart, clend)]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	706 publishing = repo.ui.configbool('phases', 'publish', True)
20966 63659b809021 phase: apply publishing enforcement for "serve" source Pierre-Yves David <pierre-yves.david@fb.com> parents: 20942 diff changeset	707 if srctype in ('push', 'serve'):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	708 # Old servers can not push the boundary themselves.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	709 # New servers won't push the boundary if changeset already
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	710 # exists locally as secret
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	711 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	712 # We should not use added here but the list of all change in
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	713 # the bundle
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	714 if publishing:
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	715 phases.advanceboundary(repo, tr, phases.public, srccontent)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	716 else:
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	717 # Those changesets have been pushed from the outside, their
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	718 # phases are going to be pushed alongside. Therefor
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	719 # `targetphase` is ignored.
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	720 phases.advanceboundary(repo, tr, phases.draft, srccontent)
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	721 phases.retractboundary(repo, tr, phases.draft, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	722 elif srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	723 # publishing only alter behavior during push
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	724 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	725 # strip should not touch boundary at all
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	726 phases.retractboundary(repo, tr, targetphase, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	727
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	728 # make changelog see real files again
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	729 cl.finalize(trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	730
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	731 tr.close()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	732
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	733 if changesets > 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	734 if srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	735 # During strip, branchcache is invalid but coming call to
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	736 # `destroyed` will repair it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	737 # In other case we can safely update cache on disk.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	738 branchmap.updatecache(repo.filtered('served'))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	739
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	740 def runhooks():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	741 # These hooks run when the lock releases, not when the
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	742 # transaction closes. So it's possible for the changelog
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	743 # to have changed since we last saw it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	744 if clstart >= len(repo):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	745 return
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	746
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	747 # forcefully update the on-disk branch cache
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	748 repo.ui.debug("updating the branch cache\n")
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	749 repo.hook("changegroup", **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	750
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	751 for n in added:
22968 01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	752 args = hookargs.copy()
01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	753 args['node'] = hex(n)
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	754 repo.hook("incoming", **args)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	755
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	756 newheads = [h for h in repo.heads() if h not in oldheads]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	757 repo.ui.log("incoming",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	758 "%s incoming changes - new heads: %s\n",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	759 len(added),
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	760 ', '.join([hex(c[:6]) for c in newheads]))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	761 repo._afterlock(runhooks)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	762
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	763 finally:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	764 tr.release()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	765 # never return 0 here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	766 if dh < 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	767 return dh - 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	768 else:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	769 return dh + 1

Mercurial > public > mercurial-scm > hg

annotate mercurial/changegroup.py @ 23381:cc0ff93d0c0c stable