mercurial-scm/hg: mercurial/changegroup.py annotate

annotate mercurial/changegroup.py @ 24120:a450e0a2ba0a

revlog: in addgroup, reject ill-formed deltas based on censored nodes To ensure interoperability when clones disagree about which file nodes are censored, a restriction is made on deltas based on censored nodes. Any such delta must replace the full text of the base in a single patch. If the recipient of a delta considers the base to be censored and the delta is not in the expected form, the recipient must reject it, as it can't know if the source has also censored the base. For background and broader design of the censorship feature, see: http://mercurial.selenic.com/wiki/CensorPlan

author	Mike Edgar <adgar@google.com>
date	Fri, 06 Feb 2015 00:55:29 +0000
parents	f99a6e1865e5
children	d8e0c591781c

rev	line source
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9437 diff changeset	6 # GNU General Public License version 2 or any later version.
3877 abaee83ce0a6 Replace demandload with new demandimport Matt Mackall <mpm@selenic.com> parents: 3859 diff changeset	7
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	8 import weakref
3891 6b4127c7d52a Simplify i18n imports Matt Mackall <mpm@selenic.com> parents: 3877 diff changeset	9 from i18n import _
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	10 from node import nullrev, nullid, hex, short
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	11 import mdiff, util, dagutil
8312 b87a50b7125c separate import lines from mercurial and general python modules Simon Heimberg <simohe@besonet.ch> parents: 8226 diff changeset	12 import struct, os, bz2, zlib, tempfile
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	13 import discovery, error, phases, branchmap
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	14
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	15 _CHANGEGROUPV1_DELTA_HEADER = "20s20s20s20s"
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	16 _CHANGEGROUPV2_DELTA_HEADER = "20s20s20s20s20s"
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	17
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	18 def readexactly(stream, n):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	19 '''read n bytes from stream.read and abort if less was available'''
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	20 s = stream.read(n)
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	21 if len(s) < n:
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	22 raise util.Abort(_("stream ended unexpectedly"
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	23 " (got %d bytes, expected %d)")
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	24 % (len(s), n))
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	25 return s
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	26
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	27 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	28 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	29 d = readexactly(stream, 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	30 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	31 if l <= 4:
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	32 if l:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	33 raise util.Abort(_("invalid chunk length %d") % l)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	34 return ""
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	35 return readexactly(stream, l - 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	36
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	37 def chunkheader(length):
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	38 """return a changegroup chunk header (string)"""
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	39 return struct.pack(">l", length + 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	40
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	41 def closechunk():
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	42 """return a changegroup chunk header (string) for a zero-length chunk"""
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	43 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	44
23890 7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	45 def combineresults(results):
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	46 """logic to combine 0 or more addchangegroup results into one"""
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	47 changedheads = 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	48 result = 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	49 for ret in results:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	50 # If any changegroup result is 0, return 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	51 if ret == 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	52 result = 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	53 break
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	54 if ret < -1:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	55 changedheads += ret + 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	56 elif ret > 1:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	57 changedheads += ret - 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	58 if changedheads > 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	59 result = 1 + changedheads
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	60 elif changedheads < 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	61 result = -1 + changedheads
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	62 return result
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	63
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	64 class nocompress(object):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	65 def compress(self, x):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	66 return x
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	67 def flush(self):
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	68 return ""
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	69
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	70 bundletypes = {
14060 aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	71 "": ("", nocompress), # only when using unbundle on ssh and old http servers
aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	72 # since the unification ssh accepts a header but there
aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	73 # is no capability signaling it.
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	74 "HG2Y": (), # special-cased below
3704 9c1737a3e254 fix writebundle for bz2 bundles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3662 diff changeset	75 "HG10UN": ("HG10UN", nocompress),
3762 b9d3e12da485 changegroup.py: delay the loading of the bz2 and zlib modules Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3706 diff changeset	76 "HG10BZ": ("HG10", lambda: bz2.BZ2Compressor()),
b9d3e12da485 changegroup.py: delay the loading of the bz2 and zlib modules Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3706 diff changeset	77 "HG10GZ": ("HG10GZ", lambda: zlib.compressobj()),
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	78 }
f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	79
9087 f48454a279b9 typos: "it's" -> "its" Martin Geisler <mg@lazybytes.net> parents: 8312 diff changeset	80 # hgweb uses this list to communicate its preferred type
6152 c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	81 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	82
23895 cda18ded2c48 changegroup.writebundle: provide ui Eric Sumner <ericsumner@fb.com> parents: 23890 diff changeset	83 def writebundle(ui, cg, filename, bundletype, vfs=None):
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	84 """Write a bundle file and return its filename.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	85
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	86 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	87 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	88 bz2 compression can be turned off.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	89 The bundle file will be deleted in case of errors.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	90 """
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	91
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	92 fh = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	93 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	94 try:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	95 if filename:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	96 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	97 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	98 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	99 fh = open(filename, "wb")
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	100 else:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	101 fd, filename = tempfile.mkstemp(prefix="hg-bundle-", suffix=".hg")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	102 fh = os.fdopen(fd, "wb")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	103 cleanup = filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	104
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	105 if bundletype == "HG2Y":
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	106 import bundle2
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	107 bundle = bundle2.bundle20(ui)
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	108 part = bundle.newpart('b2x:changegroup', data=cg.getchunks())
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	109 part.addparam('version', cg.version)
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	110 z = nocompress()
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	111 chunkiter = bundle.getchunks()
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	112 else:
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	113 if cg.version != '01':
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	114 raise util.Abort(_('Bundle1 only supports v1 changegroups\n'))
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	115 header, compressor = bundletypes[bundletype]
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	116 fh.write(header)
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	117 z = compressor()
becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	118 chunkiter = cg.getchunks()
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	119
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	120 # parse the changegroup data, otherwise we will block
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	121 # in case of sshrepo because we don't know the end of the stream
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	122
12335 e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	123 # an empty chunkgroup is the end of the changegroup
e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	124 # a changegroup has at least 2 chunkgroups (changelog and manifest).
e21fe9c5fb25 bundle: get rid of chunkiter Matt Mackall <mpm@selenic.com> parents: 12334 diff changeset	125 # after that, an empty chunkgroup is the end of the changegroup
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	126 for chunk in chunkiter:
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	127 fh.write(z.compress(chunk))
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	128 fh.write(z.flush())
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	129 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	130 return filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	131 finally:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	132 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	133 fh.close()
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	134 if cleanup is not None:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	135 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	136 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	137 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	138 os.unlink(cleanup)
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	139
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	140 def decompressor(fh, alg):
270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	141 if alg == 'UN':
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	142 return fh
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	143 elif alg == 'GZ':
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	144 def generator(f):
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	145 zd = zlib.decompressobj()
16557 9dba55369cd8 changegroup: decompress GZ algorithm in larger chunks for better performance Michael Tj?rnemark <michael@tjornemark.dk> parents: 14494 diff changeset	146 for chunk in util.filechunkiter(f):
6154 ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6152 diff changeset	147 yield zd.decompress(chunk)
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	148 elif alg == 'BZ':
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	149 def generator(f):
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	150 zd = bz2.BZ2Decompressor()
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	151 zd.decompress("BZ")
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	152 for chunk in util.filechunkiter(f, 4096):
8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	153 yield zd.decompress(chunk)
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	154 else:
270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	155 raise util.Abort("unknown bundle compression '%s'" % alg)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	156 return util.chunkbuffer(generator(fh))
12041 270fb4d39153 bundle: factor out decompressor Matt Mackall <mpm@selenic.com> parents: 11648 diff changeset	157
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	158 class cg1unpacker(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	159 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	160 deltaheadersize = struct.calcsize(deltaheader)
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	161 version = '01'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	162 def __init__(self, fh, alg):
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	163 self._stream = decompressor(fh, alg)
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	164 self._type = alg
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	165 self.callback = None
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	166 def compressed(self):
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	167 return self._type != 'UN'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	168 def read(self, l):
bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	169 return self._stream.read(l)
12330 e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	170 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	171 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	172 def tell(self):
12332 680fe77ab5b8 bundlerepo: use bundle objects everywhere Matt Mackall <mpm@selenic.com> parents: 12330 diff changeset	173 return self._stream.tell()
12347 6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	174 def close(self):
6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	175 return self._stream.close()
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	176
50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	177 def chunklength(self):
13459 acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34 Jim Hague <jim.hague@acm.org> parents: 13458 diff changeset	178 d = readexactly(self._stream, 4)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	179 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	180 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	181 if l:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	182 raise util.Abort(_("invalid chunk length %d") % l)
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	183 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	184 if self.callback:
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	185 self.callback()
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	186 return l - 4
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	187
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	188 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	189 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	190 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	191
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	192 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	193 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	194 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	195
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	196 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	197 """return the header of the filelogs chunk, v10 only has the filename"""
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	198 l = self.chunklength()
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	199 if not l:
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	200 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	201 fname = readexactly(self._stream, l)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	202 return {'filename': fname}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	203
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	204 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	205 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	206 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	207 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	208 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	209 deltabase = prevnode
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	210 return node, p1, p2, deltabase, cs
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	211
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	212 def deltachunk(self, prevnode):
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	213 l = self.chunklength()
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	214 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	215 return {}
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	216 headerdata = readexactly(self._stream, self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	217 header = struct.unpack(self.deltaheader, headerdata)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	218 delta = readexactly(self._stream, l - self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	219 node, p1, p2, deltabase, cs = self._deltaheader(header, prevnode)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	220 return {'node': node, 'p1': p1, 'p2': p2, 'cs': cs,
f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	221 'deltabase': deltabase, 'delta': delta}
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	222
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	223 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	224 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	225
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	226 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	227 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	228 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	229 """
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	230 # an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	231 # a changegroup has at least 2 chunkgroups (changelog and manifest).
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	232 # after that, an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	233 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	234 count = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	235 while not empty or count <= 2:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	236 empty = True
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	237 count += 1
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	238 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	239 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	240 if not chunk:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	241 break
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	242 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	243 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	244 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	245 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	246 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	247 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	248 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	249 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	250
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	251 class cg2unpacker(cg1unpacker):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	252 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	253 deltaheadersize = struct.calcsize(deltaheader)
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	254 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	255
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	256 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	257 node, p1, p2, deltabase, cs = headertuple
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	258 return node, p1, p2, deltabase, cs
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	259
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	260 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	261 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	262 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	263 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	264 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	265 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	266 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	267 if len(d) < n:
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	268 d += readexactly(self._fh, n - len(d))
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	269 return d
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	270 return readexactly(self._fh, n)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	271
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	272 class cg1packer(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	273 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	274 version = '01'
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	275 def __init__(self, repo, bundlecaps=None):
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	276 """Given a source repo, construct a bundler.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	277
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	278 bundlecaps is optional and can be used to specify the set of
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	279 capabilities which can be used to build the bundle.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	280 """
19201 309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	281 # Set of capabilities we can use to build the bundle.
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	282 if bundlecaps is None:
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	283 bundlecaps = set()
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	284 self._bundlecaps = bundlecaps
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	285 self._changelog = repo.changelog
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	286 self._manifest = repo.manifest
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	287 reorder = repo.ui.config('bundle', 'reorder', 'auto')
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	288 if reorder == 'auto':
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	289 reorder = None
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	290 else:
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	291 reorder = util.parsebool(reorder)
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	292 self._repo = repo
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	293 self._reorder = reorder
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	294 self._progress = repo.ui.progress
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	295 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	296 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	297 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	298 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	299
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	300 def close(self):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	301 return closechunk()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	302
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	303 def fileheader(self, fname):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	304 return chunkheader(len(fname)) + fname
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	305
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	306 def group(self, nodelist, revlog, lookup, units=None, reorder=None):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	307 """Calculate a delta group, yielding a sequence of changegroup chunks
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	308 (strings).
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	309
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	310 Given a list of changeset revs, return a set of deltas and
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	311 metadata corresponding to nodes. The first delta is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	312 first parent(nodelist[0]) -> nodelist[0], the receiver is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	313 guaranteed to have this parent as it has all history before
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	314 these changesets. In the case firstparent is nullrev the
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	315 changegroup starts with a full revision.
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	316
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	317 If units is not None, progress detail will be generated, units specifies
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	318 the type of revlog that is touched (changelog, manifest, etc.).
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	319 """
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	320 # if we don't have any revisions touched by these changesets, bail
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	321 if len(nodelist) == 0:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	322 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	323 return
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	324
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	325 # for generaldelta revlogs, we linearize the revs; this will both be
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	326 # much quicker and generate a much smaller bundle
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	327 if (revlog._generaldelta and reorder is not False) or reorder:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	328 dag = dagutil.revlogdag(revlog)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	329 revs = set(revlog.rev(n) for n in nodelist)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	330 revs = dag.linearize(revs)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	331 else:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	332 revs = sorted([revlog.rev(n) for n in nodelist])
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	333
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	334 # add the parent of the first rev
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	335 p = revlog.parentrevs(revs[0])[0]
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	336 revs.insert(0, p)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	337
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	338 # build deltas
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	339 total = len(revs) - 1
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	340 msgbundling = _('bundling')
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	341 for r in xrange(len(revs) - 1):
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	342 if units is not None:
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	343 self._progress(msgbundling, r + 1, unit=units, total=total)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	344 prev, curr = revs[r], revs[r + 1]
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	345 linknode = lookup(revlog.node(curr))
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	346 for c in self.revchunk(revlog, curr, prev, linknode):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	347 yield c
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	348
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	349 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	350
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	351 # filter any nodes that claim to be part of the known set
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	352 def prune(self, revlog, missing, commonrevs, source):
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	353 rr, rl = revlog.rev, revlog.linkrev
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	354 return [n for n in missing if rl(rr(n)) not in commonrevs]
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	355
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	356 def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	357 '''yield a sequence of changegroup chunks (strings)'''
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	358 repo = self._repo
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	359 cl = self._changelog
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	360 mf = self._manifest
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	361 reorder = self._reorder
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	362 progress = self._progress
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	363
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	364 # for progress output
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	365 msgbundling = _('bundling')
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	366
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	367 clrevorder = {}
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	368 mfs = {} # needed manifests
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	369 fnodes = {} # needed file nodes
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	370 changedfiles = set()
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	371
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	372 # Callback for the changelog, used to collect changed files and manifest
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	373 # nodes.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	374 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	375 def lookupcl(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	376 c = cl.read(x)
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	377 clrevorder[x] = len(clrevorder)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	378 changedfiles.update(c[3])
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	379 # record the first changeset introducing this manifest version
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	380 mfs.setdefault(c[0], x)
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	381 return x
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	382
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	383 self._verbosenote(_('uncompressed size of bundle content:\n'))
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	384 size = 0
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	385 for chunk in self.group(clnodes, cl, lookupcl, units=_('changesets'),
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	386 reorder=reorder):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	387 size += len(chunk)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	388 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	389 self._verbosenote(_('%8.i (changelog)\n') % size)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	390 progress(msgbundling, None)
f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	391
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	392 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	393 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	394 # Returns the linkrev node (collected in lookupcl).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	395 def lookupmf(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	396 clnode = mfs[x]
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	397 if not fastpathlinkrev or reorder:
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	398 mdata = mf.readfast(x)
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	399 for f, n in mdata.iteritems():
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	400 if f in changedfiles:
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	401 # record the first changeset introducing this filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	402 # version
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	403 fclnodes = fnodes.setdefault(f, {})
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	404 fclnode = fclnodes.setdefault(n, clnode)
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	405 if clrevorder[clnode] < clrevorder[fclnode]:
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	406 fclnodes[n] = clnode
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	407 return clnode
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	408
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	409 mfnodes = self.prune(mf, mfs, commonrevs, source)
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	410 size = 0
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	411 for chunk in self.group(mfnodes, mf, lookupmf, units=_('manifests'),
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	412 reorder=reorder):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	413 size += len(chunk)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	414 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	415 self._verbosenote(_('%8.i (manifests)\n') % size)
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	416 progress(msgbundling, None)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	417
6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	418 mfs.clear()
19708 fd4f612f7cb6 bundle: fix performance regression when bundling file changes (issue4031) Antoine Pitrou <solipsis@pitrou.net> parents: 19334 diff changeset	419 needed = set(cl.rev(x) for x in clnodes)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	420
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	421 def linknodes(filerevlog, fname):
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	422 if fastpathlinkrev and not reorder:
20936 bfb40168391c changegroup: remove unused variable caught by pyflakes Sean Farley <sean.michael.farley@gmail.com> parents: 19708 diff changeset	423 llr = filerevlog.linkrev
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	424 def genfilenodes():
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	425 for r in filerevlog:
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	426 linkrev = llr(r)
19325 ec896f9e8894 changegroup: fix fastpath during commit Matt Mackall <mpm@selenic.com> parents: 19293 diff changeset	427 if linkrev in needed:
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	428 yield filerevlog.node(r), cl.node(linkrev)
23225 bdf7b1ea1dae changegroup: don't store unused value on fnodes (issue4443) Gregory Szorc <gregory.szorc@gmail.com> parents: 23224 diff changeset	429 return dict(genfilenodes())
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	430 return fnodes.get(fname, {})
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	431
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	432 for chunk in self.generatefiles(changedfiles, linknodes, commonrevs,
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	433 source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	434 yield chunk
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	435
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	436 yield self.close()
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	437 progress(msgbundling, None)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	438
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	439 if clnodes:
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	440 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	441
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	442 def generatefiles(self, changedfiles, linknodes, commonrevs, source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	443 repo = self._repo
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	444 progress = self._progress
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	445 reorder = self._reorder
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	446 msgbundling = _('bundling')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	447
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	448 total = len(changedfiles)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	449 # for progress output
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	450 msgfiles = _('files')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	451 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	452 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	453 if not filerevlog:
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	454 raise util.Abort(_("empty or missing revlog for %s") % fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	455
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	456 linkrevnodes = linknodes(filerevlog, fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	457 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	458 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	459 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	460 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	461
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	462 filenodes = self.prune(filerevlog, linkrevnodes, commonrevs, source)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	463 if filenodes:
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	464 progress(msgbundling, i + 1, item=fname, unit=msgfiles,
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	465 total=total)
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	466 h = self.fileheader(fname)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	467 size = len(h)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	468 yield h
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	469 for chunk in self.group(filenodes, filerevlog, lookupfilelog,
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	470 reorder=reorder):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	471 size += len(chunk)
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	472 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	473 self._verbosenote(_('%8.i %s\n') % (size, fname))
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	474
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	475 def deltaparent(self, revlog, rev, p1, p2, prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	476 return prev
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	477
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	478 def revchunk(self, revlog, rev, prev, linknode):
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	479 node = revlog.node(rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	480 p1, p2 = revlog.parentrevs(rev)
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	481 base = self.deltaparent(revlog, rev, p1, p2, prev)
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	482
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	483 prefix = ''
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	484 if base == nullrev:
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	485 delta = revlog.revision(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	486 prefix = mdiff.trivialdiffheader(len(delta))
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	487 else:
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	488 delta = revlog.revdiff(base, rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	489 p1n, p2n = revlog.parents(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	490 basenode = revlog.node(base)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	491 meta = self.builddeltaheader(node, p1n, p2n, basenode, linknode)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	492 meta += prefix
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	493 l = len(meta) + len(delta)
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	494 yield chunkheader(l)
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	495 yield meta
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	496 yield delta
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	497 def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	498 # do nothing with basenode, it is implicitly the previous one in HG10
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	499 return struct.pack(self.deltaheader, node, p1n, p2n, linknode)
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	500
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	501 class cg2packer(cg1packer):
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	502 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	503 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	504
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	505 def group(self, nodelist, revlog, lookup, units=None, reorder=None):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	506 if (revlog._generaldelta and reorder is not True):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	507 reorder = False
23243 c58432685cff changegroup.cg2packer: lookup 'group' via inheritance chain Siddharth Agarwal <sid0@fb.com> parents: 23221 diff changeset	508 return super(cg2packer, self).group(nodelist, revlog, lookup,
c58432685cff changegroup.cg2packer: lookup 'group' via inheritance chain Siddharth Agarwal <sid0@fb.com> parents: 23221 diff changeset	509 units=units, reorder=reorder)
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	510
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	511 def deltaparent(self, revlog, rev, p1, p2, prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	512 dp = revlog.deltaparent(rev)
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	513 # avoid storing full revisions; pick prev in those cases
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	514 # also pick prev when we can't be sure remote has dp
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	515 if dp == nullrev or (dp != p1 and dp != p2 and dp != prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	516 return prev
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	517 return dp
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	518
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	519 def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	520 return struct.pack(self.deltaheader, node, p1n, p2n, basenode, linknode)
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	521
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	522 packermap = {'01': (cg1packer, cg1unpacker),
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	523 '02': (cg2packer, cg2unpacker)}
23168 a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions Pierre-Yves David <pierre-yves.david@fb.com> parents: 22971 diff changeset	524
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	525 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	526 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	527 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	528 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	529 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	530 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	531 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	532
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	533 def getsubsetraw(repo, outgoing, bundler, source, fastpath=False):
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	534 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	535 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	536 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	537 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	538 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	539 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	540 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	541 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	542 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	543 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	544
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	545 repo.hook('preoutgoing', throw=True, source=source)
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	546 _changegroupinfo(repo, csets, source)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	547 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	548
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	549 def getsubset(repo, outgoing, bundler, source, fastpath=False, version='01'):
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	550 gengroup = getsubsetraw(repo, outgoing, bundler, source, fastpath)
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	551 return packermap[version][1](util.chunkbuffer(gengroup), 'UN')
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	552
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	553 def changegroupsubset(repo, roots, heads, source, version='01'):
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	554 """Compute a changegroup consisting of all the nodes that are
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	555 descendants of any of the roots and ancestors of any of the heads.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	556 Return a chunkbuffer object whose read() method will return
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	557 successive changegroup chunks.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	558
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	559 It is fairly complex as determining which filenodes and which
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	560 manifest nodes need to be included for the changeset to be complete
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	561 is non-trivial.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	562
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	563 Another wrinkle is doing the reverse, figuring out which changeset in
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	564 the changegroup a particular filenode or manifestnode belongs to.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	565 """
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	566 cl = repo.changelog
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	567 if not roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	568 roots = [nullid]
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	569 # TODO: remove call to nodesbetween.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	570 csets, roots, heads = cl.nodesbetween(roots, heads)
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	571 discbases = []
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	572 for n in roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	573 discbases.extend([p for p in cl.parents(n) if p != nullid])
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	574 outgoing = discovery.outgoing(cl, discbases, heads)
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	575 bundler = packermap[version][0](repo)
f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	576 return getsubset(repo, outgoing, bundler, source, version=version)
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	577
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	578 def getlocalchangegroupraw(repo, source, outgoing, bundlecaps=None,
5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	579 version='01'):
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	580 """Like getbundle, but taking a discovery.outgoing as an argument.
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	581
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	582 This is only implemented for local repos and reuses potentially
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	583 precomputed sets in outgoing. Returns a raw changegroup generator."""
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	584 if not outgoing.missing:
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	585 return None
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	586 bundler = packermap[version][0](repo, bundlecaps)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	587 return getsubsetraw(repo, outgoing, bundler, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	588
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	589 def getlocalchangegroup(repo, source, outgoing, bundlecaps=None):
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	590 """Like getbundle, but taking a discovery.outgoing as an argument.
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	591
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	592 This is only implemented for local repos and reuses potentially
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	593 precomputed sets in outgoing."""
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	594 if not outgoing.missing:
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	595 return None
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	596 bundler = cg1packer(repo, bundlecaps)
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	597 return getsubset(repo, outgoing, bundler, source)
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	598
21260 aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	599 def _computeoutgoing(repo, heads, common):
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	600 """Computes which revs are outgoing given a set of common
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	601 and a set of heads.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	602
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	603 This is a separate function so extensions can have access to
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	604 the logic.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	605
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	606 Returns a discovery.outgoing object.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	607 """
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	608 cl = repo.changelog
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	609 if common:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	610 hasnode = cl.hasnode
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	611 common = [n for n in common if hasnode(n)]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	612 else:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	613 common = [nullid]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	614 if not heads:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	615 heads = cl.heads()
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	616 return discovery.outgoing(cl, common, heads)
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	617
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	618 def getchangegroupraw(repo, source, heads=None, common=None, bundlecaps=None,
5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	619 version='01'):
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	620 """Like changegroupsubset, but returns the set difference between the
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	621 ancestors of heads and the ancestors common.
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	622
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	623 If heads is None, use the local heads. If common is None, use [nullid].
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	624
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	625 If version is None, use a version '1' changegroup.
5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	626
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	627 The nodes in common might not all be known locally due to the way the
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	628 current discovery protocol works. Returns a raw changegroup generator.
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	629 """
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	630 outgoing = _computeoutgoing(repo, heads, common)
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	631 return getlocalchangegroupraw(repo, source, outgoing, bundlecaps=bundlecaps,
5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	632 version=version)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	633
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	634 def getchangegroup(repo, source, heads=None, common=None, bundlecaps=None):
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	635 """Like changegroupsubset, but returns the set difference between the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	636 ancestors of heads and the ancestors common.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	637
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	638 If heads is None, use the local heads. If common is None, use [nullid].
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	639
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	640 The nodes in common might not all be known locally due to the way the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	641 current discovery protocol works.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	642 """
21260 aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	643 outgoing = _computeoutgoing(repo, heads, common)
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	644 return getlocalchangegroup(repo, source, outgoing, bundlecaps=bundlecaps)
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	645
20931 de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	646 def changegroup(repo, basenodes, source):
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	647 # to avoid a race we use changegroupsubset() (issue1320)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	648 return changegroupsubset(repo, basenodes, repo.heads(), source)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	649
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	650 def addchangegroupfiles(repo, source, revmap, trp, pr, needfiles):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	651 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	652 files = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	653 while True:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	654 chunkdata = source.filelogheader()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	655 if not chunkdata:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	656 break
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	657 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	658 repo.ui.debug("adding %s revisions\n" % f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	659 pr()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	660 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	661 o = len(fl)
24120 a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	662 try:
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	663 if not fl.addgroup(source, revmap, trp):
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	664 raise util.Abort(_("received file revlog group is empty"))
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	665 except error.CensoredBaseError, e:
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	666 raise util.Abort(_("received delta base is censored: %s") % e)
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	667 revisions += len(fl) - o
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	668 files += 1
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	669 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	670 needs = needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	671 for new in xrange(o, len(fl)):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	672 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	673 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	674 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	675 else:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	676 raise util.Abort(
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	677 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	678 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	679 del needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	680 repo.ui.progress(_('files'), None)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	681
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	682 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	683 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	684 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	685 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	686 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	687 except error.LookupError:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	688 raise util.Abort(
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	689 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	690 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	691
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	692 return revisions, files
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	693
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	694 def addchangegroup(repo, source, srctype, url, emptyok=False,
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	695 targetphase=phases.draft):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	696 """Add the changegroup returned by source.read() to this repo.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	697 srctype is a string like 'push', 'pull', or 'unbundle'. url is
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	698 the URL of the repo where this changegroup is coming from.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	699
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	700 Return an integer summarizing the change to this repo:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	701 - nothing changed or no source: 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	702 - more heads than before: 1+added heads (2..n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	703 - fewer heads than before: -1-removed heads (-2..-n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	704 - number of heads stays the same: 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	705 """
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	706 repo = repo.unfiltered()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	707 def csmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	708 repo.ui.debug("add changeset %s\n" % short(x))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	709 return len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	710
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	711 def revmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	712 return cl.rev(x)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	713
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	714 if not source:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	715 return 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	716
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	717 changesets = files = revisions = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	718 efiles = set()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	719
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	720 tr = repo.transaction("\n".join([srctype, util.hidepassword(url)]))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	721 # The transaction could have been created before and already carries source
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	722 # information. In this case we use the top level data. We overwrite the
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	723 # argument because we need to use the top level value (if they exist) in
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	724 # this function.
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	725 srctype = tr.hookargs.setdefault('source', srctype)
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	726 url = tr.hookargs.setdefault('url', url)
23203 3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	727
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	728 # write changelog data to temp files so concurrent readers will not see
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	729 # inconsistent view
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	730 cl = repo.changelog
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	731 cl.delayupdate(tr)
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	732 oldheads = cl.heads()
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	733 try:
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	734 repo.hook('prechangegroup', throw=True, **tr.hookargs)
22969 805c18b27f48 addchangegroup: call `prechangegroup` hook after transaction retrieval Pierre-Yves David <pierre-yves.david@fb.com> parents: 22968 diff changeset	735
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	736 trp = weakref.proxy(tr)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	737 # pull off the changeset group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	738 repo.ui.status(_("adding changesets\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	739 clstart = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	740 class prog(object):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	741 step = _('changesets')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	742 count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	743 ui = repo.ui
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	744 total = None
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	745 def __call__(repo):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	746 repo.ui.progress(repo.step, repo.count, unit=_('chunks'),
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	747 total=repo.total)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	748 repo.count += 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	749 pr = prog()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	750 source.callback = pr
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	751
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	752 source.changelogheader()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	753 srccontent = cl.addgroup(source, csmap, trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	754 if not (srccontent or emptyok):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	755 raise util.Abort(_("received changelog group is empty"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	756 clend = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	757 changesets = clend - clstart
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	758 for c in xrange(clstart, clend):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	759 efiles.update(repo[c].files())
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	760 efiles = len(efiles)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	761 repo.ui.progress(_('changesets'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	762
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	763 # pull off the manifest group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	764 repo.ui.status(_("adding manifests\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	765 pr.step = _('manifests')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	766 pr.count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	767 pr.total = changesets # manifests <= changesets
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	768 # no need to check for empty manifest group here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	769 # if the result of the merge of 1 and 2 is the same in 3 and 4,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	770 # no new manifest will be created and the manifest group will
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	771 # be empty during the pull
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	772 source.manifestheader()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	773 repo.manifest.addgroup(source, revmap, trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	774 repo.ui.progress(_('manifests'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	775
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	776 needfiles = {}
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	777 if repo.ui.configbool('server', 'validate', default=False):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	778 # validate incoming csets have their manifests
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	779 for cset in xrange(clstart, clend):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	780 mfest = repo.changelog.read(repo.changelog.node(cset))[0]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	781 mfest = repo.manifest.readdelta(mfest)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	782 # store file nodes we must see
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	783 for f, n in mfest.iteritems():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	784 needfiles.setdefault(f, set()).add(n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	785
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	786 # process the files
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	787 repo.ui.status(_("adding file changes\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	788 pr.step = _('files')
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	789 pr.count = 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	790 pr.total = efiles
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	791 source.callback = None
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	792
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	793 newrevs, newfiles = addchangegroupfiles(repo, source, revmap, trp, pr,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	794 needfiles)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	795 revisions += newrevs
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	796 files += newfiles
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	797
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	798 dh = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	799 if oldheads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	800 heads = cl.heads()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	801 dh = len(heads) - len(oldheads)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	802 for h in heads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	803 if h not in oldheads and repo[h].closesbranch():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	804 dh -= 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	805 htext = ""
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	806 if dh:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	807 htext = _(" (%+d heads)") % dh
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	808
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	809 repo.ui.status(_("added %d changesets"
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	810 " with %d changes to %d files%s\n")
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	811 % (changesets, revisions, files, htext))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	812 repo.invalidatevolatilesets()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	813
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	814 if changesets > 0:
23203 3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	815 p = lambda: tr.writepending() and repo.root or ""
21151 240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	816 if 'node' not in tr.hookargs:
240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	817 tr.hookargs['node'] = hex(cl.node(clstart))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	818 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	819 else:
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	820 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	821 hookargs['node'] = hex(cl.node(clstart))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	822 repo.hook('pretxnchangegroup', throw=True, pending=p, **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	823
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	824 added = [cl.node(r) for r in xrange(clstart, clend)]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	825 publishing = repo.ui.configbool('phases', 'publish', True)
20966 63659b809021 phase: apply publishing enforcement for "serve" source Pierre-Yves David <pierre-yves.david@fb.com> parents: 20942 diff changeset	826 if srctype in ('push', 'serve'):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	827 # Old servers can not push the boundary themselves.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	828 # New servers won't push the boundary if changeset already
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	829 # exists locally as secret
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	830 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	831 # We should not use added here but the list of all change in
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	832 # the bundle
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	833 if publishing:
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	834 phases.advanceboundary(repo, tr, phases.public, srccontent)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	835 else:
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	836 # Those changesets have been pushed from the outside, their
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	837 # phases are going to be pushed alongside. Therefor
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	838 # `targetphase` is ignored.
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	839 phases.advanceboundary(repo, tr, phases.draft, srccontent)
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	840 phases.retractboundary(repo, tr, phases.draft, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	841 elif srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	842 # publishing only alter behavior during push
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	843 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	844 # strip should not touch boundary at all
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	845 phases.retractboundary(repo, tr, targetphase, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	846
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	847 if changesets > 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	848 if srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	849 # During strip, branchcache is invalid but coming call to
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	850 # `destroyed` will repair it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	851 # In other case we can safely update cache on disk.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	852 branchmap.updatecache(repo.filtered('served'))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	853
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	854 def runhooks():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	855 # These hooks run when the lock releases, not when the
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	856 # transaction closes. So it's possible for the changelog
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	857 # to have changed since we last saw it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	858 if clstart >= len(repo):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	859 return
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	860
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	861 # forcefully update the on-disk branch cache
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	862 repo.ui.debug("updating the branch cache\n")
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	863 repo.hook("changegroup", **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	864
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	865 for n in added:
22968 01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	866 args = hookargs.copy()
01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	867 args['node'] = hex(n)
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	868 repo.hook("incoming", **args)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	869
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	870 newheads = [h for h in repo.heads() if h not in oldheads]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	871 repo.ui.log("incoming",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	872 "%s incoming changes - new heads: %s\n",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	873 len(added),
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	874 ', '.join([hex(c[:6]) for c in newheads]))
23221 cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	875
cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	876 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
23282 6c1351352b6c transaction: pass the transaction to 'postclose' callback Pierre-Yves David <pierre-yves.david@fb.com> parents: 23244 diff changeset	877 lambda tr: repo._afterlock(runhooks))
23221 cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	878
cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	879 tr.close()
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	880
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	881 finally:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	882 tr.release()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	883 # never return 0 here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	884 if dh < 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	885 return dh - 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	886 else:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	887 return dh + 1

Mercurial > public > mercurial-scm > hg

annotate mercurial/changegroup.py @ 24120:a450e0a2ba0a