mercurial-scm/hg: mercurial/changegroup.py annotate

annotate mercurial/changegroup.py @ 26595:be0489770925

getsubset: get the unpacker version from the bundler The current setup requires to pass both a packer and, optionally, the version of the unpacker. This is confusing and error prone as the two value cannot mismatch. Instead, we simply grab the version from the packer. This fixes a bug where requesting a cg2 from 'hg bundle' were reported as changegroup 1. I should have caught that in the initial changeset but I missed it somehow.

author	Pierre-Yves David <pierre-yves.david@fb.com>
date	Fri, 09 Oct 2015 14:59:37 -0700
parents	56b2bcea2529
children	c2e6e3cc7cb4

rev	line source
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9437 diff changeset	6 # GNU General Public License version 2 or any later version.
3877 abaee83ce0a6 Replace demandload with new demandimport Matt Mackall <mpm@selenic.com> parents: 3859 diff changeset	7
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	9
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	10 import os
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	11 import struct
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	12 import tempfile
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	13 import weakref
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	14
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	15 from .i18n import _
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	16 from .node import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	17 hex,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	18 nullid,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	19 nullrev,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	20 short,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	21 )
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	22
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	23 from . import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	24 branchmap,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	25 dagutil,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	26 discovery,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	27 error,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	28 mdiff,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	29 phases,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	30 util,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	31 )
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	32
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	33 _CHANGEGROUPV1_DELTA_HEADER = "20s20s20s20s"
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	34 _CHANGEGROUPV2_DELTA_HEADER = "20s20s20s20s20s"
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	35
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	36 def readexactly(stream, n):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	37 '''read n bytes from stream.read and abort if less was available'''
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	38 s = stream.read(n)
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	39 if len(s) < n:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	40 raise error.Abort(_("stream ended unexpectedly"
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	41 " (got %d bytes, expected %d)")
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	42 % (len(s), n))
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	43 return s
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	44
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	45 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	46 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	47 d = readexactly(stream, 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	48 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	49 if l <= 4:
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	50 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	51 raise error.Abort(_("invalid chunk length %d") % l)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	52 return ""
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	53 return readexactly(stream, l - 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	54
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	55 def chunkheader(length):
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	56 """return a changegroup chunk header (string)"""
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	57 return struct.pack(">l", length + 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	58
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	59 def closechunk():
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	60 """return a changegroup chunk header (string) for a zero-length chunk"""
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	61 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	62
23890 7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	63 def combineresults(results):
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	64 """logic to combine 0 or more addchangegroup results into one"""
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	65 changedheads = 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	66 result = 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	67 for ret in results:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	68 # If any changegroup result is 0, return 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	69 if ret == 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	70 result = 0
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	71 break
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	72 if ret < -1:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	73 changedheads += ret + 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	74 elif ret > 1:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	75 changedheads += ret - 1
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	76 if changedheads > 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	77 result = 1 + changedheads
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	78 elif changedheads < 0:
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	79 result = -1 + changedheads
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	80 return result
7817059917d0 pullbundle2: extract addchangegroup result combining into its own function Eric Sumner <ericsumner@fb.com> parents: 23748 diff changeset	81
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	82 bundletypes = {
26272 59c410db8c68 readbundle: fix typo of None compression Yuya Nishihara <yuya@tcha.org> parents: 26271 diff changeset	83 "": ("", None), # only when using unbundle on ssh and old http servers
14060 aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	84 # since the unification ssh accepts a header but there
aaa9a5989405 bundle: more comments about the different header types, remove useless if Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 13831 diff changeset	85 # is no capability signaling it.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24190 diff changeset	86 "HG20": (), # special-cased below
26271 a0eff7ebc2ae readbundle: map 'HG10UN' to None compression Pierre-Yves David <pierre-yves.david@fb.com> parents: 26270 diff changeset	87 "HG10UN": ("HG10UN", None),
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	88 "HG10BZ": ("HG10", 'BZ'),
1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	89 "HG10GZ": ("HG10GZ", 'GZ'),
3662 f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	90 }
f4dc02d7fb71 unduplicate bundle writing code from httprepo Matt Mackall <mpm@selenic.com> parents: 3660 diff changeset	91
9087 f48454a279b9 typos: "it's" -> "its" Martin Geisler <mg@lazybytes.net> parents: 8312 diff changeset	92 # hgweb uses this list to communicate its preferred type
6152 c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	93 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
c050548307a4 hgweb: use bundletypes from mercurial.changegroup Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 5906 diff changeset	94
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	95 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	96 """Write chunks to a file and return its filename.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	97
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	98 The stream is assumed to be a bundle file.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	99 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	100 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	101 """
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	102 fh = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	103 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	104 try:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	105 if filename:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	106 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	107 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	108 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	109 fh = open(filename, "wb")
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	110 else:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	111 fd, filename = tempfile.mkstemp(prefix="hg-bundle-", suffix=".hg")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	112 fh = os.fdopen(fd, "wb")
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	113 cleanup = filename
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	114 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	115 fh.write(c)
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	116 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	117 return filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	118 finally:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	119 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	120 fh.close()
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	121 if cleanup is not None:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	122 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	123 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	124 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	125 os.unlink(cleanup)
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	126
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	127 def writebundle(ui, cg, filename, bundletype, vfs=None, compression=None):
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	128 """Write a bundle file and return its filename.
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	129
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	130 Existing files will not be overwritten.
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	131 If no filename is specified, a temporary file is created.
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	132 bz2 compression can be turned off.
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	133 The bundle file will be deleted in case of errors.
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	134 """
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	135
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	136 if bundletype == "HG20":
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	137 from . import bundle2
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	138 bundle = bundle2.bundle20(ui)
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	139 bundle.setcompression(compression)
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	140 part = bundle.newpart('changegroup', data=cg.getchunks())
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	141 part.addparam('version', cg.version)
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	142 chunkiter = bundle.getchunks()
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	143 else:
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	144 # compression argument is only for the bundle2 case
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	145 assert compression is None
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	146 if cg.version != '01':
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	147 raise error.Abort(_('old bundle types only supports v1 '
56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	148 'changegroups'))
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	149 header, comp = bundletypes[bundletype]
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	150 if comp not in util.compressors:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	151 raise error.Abort(_('unknown stream compression type: %s')
56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	152 % comp)
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	153 z = util.compressors[comp]()
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	154 subchunkiter = cg.getchunks()
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	155 def chunkiter():
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	156 yield header
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	157 for chunk in subchunkiter:
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	158 yield z.compress(chunk)
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	159 yield z.flush()
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	160 chunkiter = chunkiter()
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	161
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	162 # parse the changegroup data, otherwise we will block
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	163 # in case of sshrepo because we don't know the end of the stream
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	164
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	165 # an empty chunkgroup is the end of the changegroup
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	166 # a changegroup has at least 2 chunkgroups (changelog and manifest).
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	167 # after that, an empty chunkgroup is the end of the changegroup
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	168 return writechunks(ui, chunkiter, filename, vfs=vfs)
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	169
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	170 class cg1unpacker(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	171 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	172 deltaheadersize = struct.calcsize(deltaheader)
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	173 version = '01'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	174 def __init__(self, fh, alg):
26267 eca468b8fae4 compression: use 'None' for no-compression Pierre-Yves David <pierre-yves.david@fb.com> parents: 26266 diff changeset	175 if alg == 'UN':
eca468b8fae4 compression: use 'None' for no-compression Pierre-Yves David <pierre-yves.david@fb.com> parents: 26266 diff changeset	176 alg = None # get more modern without breaking too much
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	177 if not alg in util.decompressors:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	178 raise error.Abort(_('unknown stream compression type: %s')
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	179 % alg)
26392 127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	180 if alg == 'BZ':
127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	181 alg = '_truncatedBZ'
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	182 self._stream = util.decompressors[alg](fh)
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	183 self._type = alg
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	184 self.callback = None
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	185 def compressed(self):
26267 eca468b8fae4 compression: use 'None' for no-compression Pierre-Yves David <pierre-yves.david@fb.com> parents: 26266 diff changeset	186 return self._type is not None
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	187 def read(self, l):
bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	188 return self._stream.read(l)
12330 e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	189 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	190 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	191 def tell(self):
12332 680fe77ab5b8 bundlerepo: use bundle objects everywhere Matt Mackall <mpm@selenic.com> parents: 12330 diff changeset	192 return self._stream.tell()
12347 6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	193 def close(self):
6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	194 return self._stream.close()
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	195
50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	196 def chunklength(self):
13459 acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34 Jim Hague <jim.hague@acm.org> parents: 13458 diff changeset	197 d = readexactly(self._stream, 4)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	198 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	199 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	200 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	201 raise error.Abort(_("invalid chunk length %d") % l)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	202 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	203 if self.callback:
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	204 self.callback()
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	205 return l - 4
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	206
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	207 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	208 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	209 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	210
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	211 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	212 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	213 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	214
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	215 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	216 """return the header of the filelogs chunk, v10 only has the filename"""
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	217 l = self.chunklength()
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	218 if not l:
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	219 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	220 fname = readexactly(self._stream, l)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	221 return {'filename': fname}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	222
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	223 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	224 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	225 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	226 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	227 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	228 deltabase = prevnode
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	229 return node, p1, p2, deltabase, cs
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	230
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	231 def deltachunk(self, prevnode):
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	232 l = self.chunklength()
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	233 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	234 return {}
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	235 headerdata = readexactly(self._stream, self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	236 header = struct.unpack(self.deltaheader, headerdata)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	237 delta = readexactly(self._stream, l - self.deltaheadersize)
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	238 node, p1, p2, deltabase, cs = self._deltaheader(header, prevnode)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	239 return {'node': node, 'p1': p1, 'p2': p2, 'cs': cs,
f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	240 'deltabase': deltabase, 'delta': delta}
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	241
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	242 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	243 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	244
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	245 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	246 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	247 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	248 """
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	249 # an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	250 # a changegroup has at least 2 chunkgroups (changelog and manifest).
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	251 # after that, an empty chunkgroup is the end of the changegroup
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	252 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	253 count = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	254 while not empty or count <= 2:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	255 empty = True
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	256 count += 1
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	257 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	258 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	259 if not chunk:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	260 break
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	261 empty = False
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	262 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	263 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	264 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	265 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	266 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	267 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	268 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	269
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	270 class cg2unpacker(cg1unpacker):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	271 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	272 deltaheadersize = struct.calcsize(deltaheader)
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	273 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	274
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	275 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	276 node, p1, p2, deltabase, cs = headertuple
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	277 return node, p1, p2, deltabase, cs
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	278
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	279 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	280 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	281 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	282 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	283 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	284 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	285 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	286 if len(d) < n:
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	287 d += readexactly(self._fh, n - len(d))
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	288 return d
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	289 return readexactly(self._fh, n)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	290
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	291 class cg1packer(object):
e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	292 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	293 version = '01'
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	294 def __init__(self, repo, bundlecaps=None):
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	295 """Given a source repo, construct a bundler.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	296
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	297 bundlecaps is optional and can be used to specify the set of
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	298 capabilities which can be used to build the bundle.
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	299 """
19201 309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	300 # Set of capabilities we can use to build the bundle.
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	301 if bundlecaps is None:
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	302 bundlecaps = set()
309c439cdbaa bundle-ng: add bundlecaps argument to getbundle() command Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19200 diff changeset	303 self._bundlecaps = bundlecaps
25831 578fc97904da generaldelta: mark experimental reordering option Matt Mackall <mpm@selenic.com> parents: 25823 diff changeset	304 # experimental config: bundle.reorder
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	305 reorder = repo.ui.config('bundle', 'reorder', 'auto')
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	306 if reorder == 'auto':
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	307 reorder = None
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	308 else:
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	309 reorder = util.parsebool(reorder)
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	310 self._repo = repo
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	311 self._reorder = reorder
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	312 self._progress = repo.ui.progress
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	313 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	314 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	315 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	316 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	317
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	318 def close(self):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	319 return closechunk()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	320
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	321 def fileheader(self, fname):
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	322 return chunkheader(len(fname)) + fname
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	323
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	324 def group(self, nodelist, revlog, lookup, units=None):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	325 """Calculate a delta group, yielding a sequence of changegroup chunks
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	326 (strings).
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	327
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	328 Given a list of changeset revs, return a set of deltas and
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	329 metadata corresponding to nodes. The first delta is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	330 first parent(nodelist[0]) -> nodelist[0], the receiver is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	331 guaranteed to have this parent as it has all history before
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	332 these changesets. In the case firstparent is nullrev the
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	333 changegroup starts with a full revision.
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	334
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	335 If units is not None, progress detail will be generated, units specifies
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	336 the type of revlog that is touched (changelog, manifest, etc.).
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	337 """
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	338 # if we don't have any revisions touched by these changesets, bail
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	339 if len(nodelist) == 0:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	340 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	341 return
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	342
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	343 # for generaldelta revlogs, we linearize the revs; this will both be
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	344 # much quicker and generate a much smaller bundle
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	345 if (revlog._generaldelta and self._reorder is None) or self._reorder:
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	346 dag = dagutil.revlogdag(revlog)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	347 revs = set(revlog.rev(n) for n in nodelist)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	348 revs = dag.linearize(revs)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	349 else:
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	350 revs = sorted([revlog.rev(n) for n in nodelist])
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	351
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	352 # add the parent of the first rev
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	353 p = revlog.parentrevs(revs[0])[0]
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	354 revs.insert(0, p)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	355
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	356 # build deltas
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	357 total = len(revs) - 1
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	358 msgbundling = _('bundling')
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	359 for r in xrange(len(revs) - 1):
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	360 if units is not None:
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	361 self._progress(msgbundling, r + 1, unit=units, total=total)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	362 prev, curr = revs[r], revs[r + 1]
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	363 linknode = lookup(revlog.node(curr))
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	364 for c in self.revchunk(revlog, curr, prev, linknode):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	365 yield c
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	366
24901 e9edd53770fb changegroup: close progress in same function as it's started Martin von Zweigbergk <martinvonz@google.com> parents: 24900 diff changeset	367 if units is not None:
e9edd53770fb changegroup: close progress in same function as it's started Martin von Zweigbergk <martinvonz@google.com> parents: 24900 diff changeset	368 self._progress(msgbundling, None)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	369 yield self.close()
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	370
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	371 # filter any nodes that claim to be part of the known set
24896 9183cb6886ef changegroup: removed unused 'source' parameter from prune() Martin von Zweigbergk <martinvonz@google.com> parents: 24717 diff changeset	372 def prune(self, revlog, missing, commonrevs):
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	373 rr, rl = revlog.rev, revlog.linkrev
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	374 return [n for n in missing if rl(rr(n)) not in commonrevs]
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	375
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	376 def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	377 '''yield a sequence of changegroup chunks (strings)'''
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	378 repo = self._repo
24978 f52560c64953 changegroup: drop _changelog and _manifest properties Martin von Zweigbergk <martinvonz@google.com> parents: 24977 diff changeset	379 cl = repo.changelog
f52560c64953 changegroup: drop _changelog and _manifest properties Martin von Zweigbergk <martinvonz@google.com> parents: 24977 diff changeset	380 ml = repo.manifest
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	381
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	382 clrevorder = {}
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	383 mfs = {} # needed manifests
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	384 fnodes = {} # needed file nodes
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	385 changedfiles = set()
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	386
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	387 # Callback for the changelog, used to collect changed files and manifest
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	388 # nodes.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	389 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	390 def lookupcl(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	391 c = cl.read(x)
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	392 clrevorder[x] = len(clrevorder)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	393 changedfiles.update(c[3])
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	394 # record the first changeset introducing this manifest version
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	395 mfs.setdefault(c[0], x)
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	396 return x
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	397
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	398 self._verbosenote(_('uncompressed size of bundle content:\n'))
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	399 size = 0
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	400 for chunk in self.group(clnodes, cl, lookupcl, units=_('changesets')):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	401 size += len(chunk)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	402 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	403 self._verbosenote(_('%8.i (changelog)\n') % size)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	404
24977 4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	405 # We need to make sure that the linkrev in the changegroup refers to
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	406 # the first changeset that introduced the manifest or file revision.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	407 # The fastpath is usually safer than the slowpath, because the filelogs
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	408 # are walked in revlog order.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	409 #
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	410 # When taking the slowpath with reorder=None and the manifest revlog
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	411 # uses generaldelta, the manifest may be walked in the "wrong" order.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	412 # Without 'clrevorder', we would get an incorrect linkrev (see fix in
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	413 # cc0ff93d0c0c).
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	414 #
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	415 # When taking the fastpath, we are only vulnerable to reordering
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	416 # of the changelog itself. The changelog never uses generaldelta, so
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	417 # it is only reordered when reorder=True. To handle this case, we
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	418 # simply take the slowpath, which already has the 'clrevorder' logic.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	419 # This was also fixed in cc0ff93d0c0c.
24976 147d8892fc4b changegroup: extract condition for linkrev fastpath Martin von Zweigbergk <martinvonz@google.com> parents: 24912 diff changeset	420 fastpathlinkrev = fastpathlinkrev and not self._reorder
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	421 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	422 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	423 # Returns the linkrev node (collected in lookupcl).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	424 def lookupmf(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	425 clnode = mfs[x]
24976 147d8892fc4b changegroup: extract condition for linkrev fastpath Martin von Zweigbergk <martinvonz@google.com> parents: 24912 diff changeset	426 if not fastpathlinkrev:
24899 cf1ea7566b20 changegroup: rename 'mf' to 'ml' to match 'cl', since it's a revlog Martin von Zweigbergk <martinvonz@google.com> parents: 24898 diff changeset	427 mdata = ml.readfast(x)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	428 for f, n in mdata.iteritems():
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	429 if f in changedfiles:
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	430 # record the first changeset introducing this filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	431 # version
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	432 fclnodes = fnodes.setdefault(f, {})
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	433 fclnode = fclnodes.setdefault(n, clnode)
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	434 if clrevorder[clnode] < clrevorder[fclnode]:
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	435 fclnodes[n] = clnode
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	436 return clnode
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	437
24899 cf1ea7566b20 changegroup: rename 'mf' to 'ml' to match 'cl', since it's a revlog Martin von Zweigbergk <martinvonz@google.com> parents: 24898 diff changeset	438 mfnodes = self.prune(ml, mfs, commonrevs)
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	439 size = 0
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	440 for chunk in self.group(mfnodes, ml, lookupmf, units=_('manifests')):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	441 size += len(chunk)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	442 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	443 self._verbosenote(_('%8.i (manifests)\n') % size)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	444
6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	445 mfs.clear()
24898 0bbf061564cf changegroup: rename 'needed' to 'clrevs' to match 'clnodes' Martin von Zweigbergk <martinvonz@google.com> parents: 24897 diff changeset	446 clrevs = set(cl.rev(x) for x in clnodes)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	447
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	448 def linknodes(filerevlog, fname):
24976 147d8892fc4b changegroup: extract condition for linkrev fastpath Martin von Zweigbergk <martinvonz@google.com> parents: 24912 diff changeset	449 if fastpathlinkrev:
20936 bfb40168391c changegroup: remove unused variable caught by pyflakes Sean Farley <sean.michael.farley@gmail.com> parents: 19708 diff changeset	450 llr = filerevlog.linkrev
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	451 def genfilenodes():
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	452 for r in filerevlog:
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	453 linkrev = llr(r)
24898 0bbf061564cf changegroup: rename 'needed' to 'clrevs' to match 'clnodes' Martin von Zweigbergk <martinvonz@google.com> parents: 24897 diff changeset	454 if linkrev in clrevs:
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	455 yield filerevlog.node(r), cl.node(linkrev)
23225 bdf7b1ea1dae changegroup: don't store unused value on fnodes (issue4443) Gregory Szorc <gregory.szorc@gmail.com> parents: 23224 diff changeset	456 return dict(genfilenodes())
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	457 return fnodes.get(fname, {})
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	458
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	459 for chunk in self.generatefiles(changedfiles, linknodes, commonrevs,
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	460 source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	461 yield chunk
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	462
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	463 yield self.close()
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	464
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	465 if clnodes:
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	466 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	467
24897 5c35a6040352 changegroup: document that 'source' parameter exists for extensions Martin von Zweigbergk <martinvonz@google.com> parents: 24896 diff changeset	468 # The 'source' parameter is useful for extensions
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	469 def generatefiles(self, changedfiles, linknodes, commonrevs, source):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	470 repo = self._repo
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	471 progress = self._progress
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	472 msgbundling = _('bundling')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	473
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	474 total = len(changedfiles)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	475 # for progress output
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	476 msgfiles = _('files')
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	477 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	478 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	479 if not filerevlog:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	480 raise error.Abort(_("empty or missing revlog for %s") % fname)
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	481
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	482 linkrevnodes = linknodes(filerevlog, fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	483 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	484 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	485 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	486 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	487
24896 9183cb6886ef changegroup: removed unused 'source' parameter from prune() Martin von Zweigbergk <martinvonz@google.com> parents: 24717 diff changeset	488 filenodes = self.prune(filerevlog, linkrevnodes, commonrevs)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	489 if filenodes:
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	490 progress(msgbundling, i + 1, item=fname, unit=msgfiles,
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	491 total=total)
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	492 h = self.fileheader(fname)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	493 size = len(h)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	494 yield h
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	495 for chunk in self.group(filenodes, filerevlog, lookupfilelog):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	496 size += len(chunk)
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	497 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	498 self._verbosenote(_('%8.i %s\n') % (size, fname))
24901 e9edd53770fb changegroup: close progress in same function as it's started Martin von Zweigbergk <martinvonz@google.com> parents: 24900 diff changeset	499 progress(msgbundling, None)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	500
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	501 def deltaparent(self, revlog, rev, p1, p2, prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	502 return prev
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	503
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	504 def revchunk(self, revlog, rev, prev, linknode):
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	505 node = revlog.node(rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	506 p1, p2 = revlog.parentrevs(rev)
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	507 base = self.deltaparent(revlog, rev, p1, p2, prev)
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	508
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	509 prefix = ''
24190 903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	510 if revlog.iscensored(base) or revlog.iscensored(rev):
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	511 try:
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	512 delta = revlog.revision(node)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25624 diff changeset	513 except error.CensoredNodeError as e:
24190 903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	514 delta = e.tombstone
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	515 if base == nullrev:
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	516 prefix = mdiff.trivialdiffheader(len(delta))
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	517 else:
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	518 baselen = revlog.rawsize(base)
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	519 prefix = mdiff.replacediffheader(baselen, len(delta))
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	520 elif base == nullrev:
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	521 delta = revlog.revision(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	522 prefix = mdiff.trivialdiffheader(len(delta))
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	523 else:
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	524 delta = revlog.revdiff(base, rev)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	525 p1n, p2n = revlog.parents(node)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	526 basenode = revlog.node(base)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	527 meta = self.builddeltaheader(node, p1n, p2n, basenode, linknode)
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	528 meta += prefix
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	529 l = len(meta) + len(delta)
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	530 yield chunkheader(l)
d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	531 yield meta
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	532 yield delta
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	533 def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	534 # do nothing with basenode, it is implicitly the previous one in HG10
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	535 return struct.pack(self.deltaheader, node, p1n, p2n, linknode)
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	536
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	537 class cg2packer(cg1packer):
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	538 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	539 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	540
24911 5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	541 def __init__(self, repo, bundlecaps=None):
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	542 super(cg2packer, self).__init__(repo, bundlecaps)
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	543 if self._reorder is None:
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	544 # Since generaldelta is directly supported by cg2, reordering
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	545 # generally doesn't help, so we disable it by default (treating
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	546 # bundle.reorder=auto just like bundle.reorder=False).
5447b8523fef cg2packer: set reorder=False in __init__ instead of in group() Martin von Zweigbergk <martinvonz@google.com> parents: 24910 diff changeset	547 self._reorder = False
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	548
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	549 def deltaparent(self, revlog, rev, p1, p2, prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	550 dp = revlog.deltaparent(rev)
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	551 # avoid storing full revisions; pick prev in those cases
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	552 # also pick prev when we can't be sure remote has dp
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	553 if dp == nullrev or (dp != p1 and dp != p2 and dp != prev):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	554 return prev
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	555 return dp
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	556
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	557 def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	558 return struct.pack(self.deltaheader, node, p1n, p2n, basenode, linknode)
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	559
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	560 packermap = {'01': (cg1packer, cg1unpacker),
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	561 '02': (cg2packer, cg2unpacker)}
23168 a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions Pierre-Yves David <pierre-yves.david@fb.com> parents: 22971 diff changeset	562
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	563 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	564 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	565 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	566 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	567 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	568 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	569 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	570
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	571 def getsubsetraw(repo, outgoing, bundler, source, fastpath=False):
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	572 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	573 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	574 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	575 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	576 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	577 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	578 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	579 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	580 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	581 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	582
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	583 repo.hook('preoutgoing', throw=True, source=source)
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	584 _changegroupinfo(repo, csets, source)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	585 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	586
26595 be0489770925 getsubset: get the unpacker version from the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	587 def getsubset(repo, outgoing, bundler, source, fastpath=False):
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	588 gengroup = getsubsetraw(repo, outgoing, bundler, source, fastpath)
26595 be0489770925 getsubset: get the unpacker version from the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	589 return packermap[bundler.version][1](util.chunkbuffer(gengroup), None)
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	590
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	591 def changegroupsubset(repo, roots, heads, source, version='01'):
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	592 """Compute a changegroup consisting of all the nodes that are
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	593 descendants of any of the roots and ancestors of any of the heads.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	594 Return a chunkbuffer object whose read() method will return
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	595 successive changegroup chunks.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	596
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	597 It is fairly complex as determining which filenodes and which
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	598 manifest nodes need to be included for the changeset to be complete
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	599 is non-trivial.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	600
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	601 Another wrinkle is doing the reverse, figuring out which changeset in
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	602 the changegroup a particular filenode or manifestnode belongs to.
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	603 """
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	604 cl = repo.changelog
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	605 if not roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	606 roots = [nullid]
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	607 discbases = []
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	608 for n in roots:
24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	609 discbases.extend([p for p in cl.parents(n) if p != nullid])
25677 af5b2f4ed594 changegroup: properly compute common base in changeggroupsubset (issue4736) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24717 diff changeset	610 # TODO: remove call to nodesbetween.
af5b2f4ed594 changegroup: properly compute common base in changeggroupsubset (issue4736) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24717 diff changeset	611 csets, roots, heads = cl.nodesbetween(roots, heads)
af5b2f4ed594 changegroup: properly compute common base in changeggroupsubset (issue4736) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24717 diff changeset	612 included = set(csets)
af5b2f4ed594 changegroup: properly compute common base in changeggroupsubset (issue4736) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24717 diff changeset	613 discbases = [n for n in discbases if n not in included]
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	614 outgoing = discovery.outgoing(cl, discbases, heads)
23897 f99a6e1865e5 changegroup.getsubset: support multiple versions Eric Sumner <ericsumner@fb.com> parents: 23896 diff changeset	615 bundler = packermap[version][0](repo)
26595 be0489770925 getsubset: get the unpacker version from the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	616 return getsubset(repo, outgoing, bundler, source)
20927 24a443948627 localrepo: move the changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20926 diff changeset	617
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	618 def getlocalchangegroupraw(repo, source, outgoing, bundlecaps=None,
5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	619 version='01'):
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	620 """Like getbundle, but taking a discovery.outgoing as an argument.
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	621
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	622 This is only implemented for local repos and reuses potentially
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	623 precomputed sets in outgoing. Returns a raw changegroup generator."""
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	624 if not outgoing.missing:
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	625 return None
23178 5e895ed5e955 changegroup: allow use of different cg#packer in getchangegroupraw Sune Foldager <cryo@cyanite.org> parents: 23177 diff changeset	626 bundler = packermap[version][0](repo, bundlecaps)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	627 return getsubsetraw(repo, outgoing, bundler, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	628
26508 47a12de9ac3f changegroup: add version argument to getlocalchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26424 diff changeset	629 def getlocalchangegroup(repo, source, outgoing, bundlecaps=None,
47a12de9ac3f changegroup: add version argument to getlocalchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26424 diff changeset	630 version='01'):
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	631 """Like getbundle, but taking a discovery.outgoing as an argument.
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	632
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	633 This is only implemented for local repos and reuses potentially
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	634 precomputed sets in outgoing."""
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	635 if not outgoing.missing:
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	636 return None
26508 47a12de9ac3f changegroup: add version argument to getlocalchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26424 diff changeset	637 bundler = packermap[version][0](repo, bundlecaps)
20928 91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	638 return getsubset(repo, outgoing, bundler, source)
91b47139d0cb localrepo: move the getlocalbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20927 diff changeset	639
25400 7759dc97c5c7 changegroup: rename _computeoutgoing to computeoutgoing Gregory Szorc <gregory.szorc@gmail.com> parents: 24978 diff changeset	640 def computeoutgoing(repo, heads, common):
21260 aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	641 """Computes which revs are outgoing given a set of common
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	642 and a set of heads.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	643
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	644 This is a separate function so extensions can have access to
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	645 the logic.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	646
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	647 Returns a discovery.outgoing object.
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	648 """
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	649 cl = repo.changelog
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	650 if common:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	651 hasnode = cl.hasnode
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	652 common = [n for n in common if hasnode(n)]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	653 else:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	654 common = [nullid]
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	655 if not heads:
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	656 heads = cl.heads()
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	657 return discovery.outgoing(cl, common, heads)
aa3e56607675 changegroup: refactor outgoing logic into a function Durham Goode <durham@fb.com> parents: 21153 diff changeset	658
26509 83d82fbefccb changegroup: add version argument to getchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26508 diff changeset	659 def getchangegroup(repo, source, heads=None, common=None, bundlecaps=None,
83d82fbefccb changegroup: add version argument to getchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26508 diff changeset	660 version='01'):
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	661 """Like changegroupsubset, but returns the set difference between the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	662 ancestors of heads and the ancestors common.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	663
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	664 If heads is None, use the local heads. If common is None, use [nullid].
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	665
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	666 The nodes in common might not all be known locally due to the way the
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	667 current discovery protocol works.
4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	668 """
25400 7759dc97c5c7 changegroup: rename _computeoutgoing to computeoutgoing Gregory Szorc <gregory.szorc@gmail.com> parents: 24978 diff changeset	669 outgoing = computeoutgoing(repo, heads, common)
26509 83d82fbefccb changegroup: add version argument to getchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26508 diff changeset	670 return getlocalchangegroup(repo, source, outgoing, bundlecaps=bundlecaps,
83d82fbefccb changegroup: add version argument to getchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 26508 diff changeset	671 version=version)
20930 4a987060d97e localrepo: move the getbundle method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20928 diff changeset	672
20931 de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	673 def changegroup(repo, basenodes, source):
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	674 # to avoid a race we use changegroupsubset() (issue1320)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	675 return changegroupsubset(repo, basenodes, repo.heads(), source)
de60ca3a390e localrepo: move the changegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20930 diff changeset	676
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	677 def addchangegroupfiles(repo, source, revmap, trp, pr, needfiles):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	678 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	679 files = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	680 while True:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	681 chunkdata = source.filelogheader()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	682 if not chunkdata:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	683 break
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	684 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	685 repo.ui.debug("adding %s revisions\n" % f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	686 pr()
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	687 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	688 o = len(fl)
24120 a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	689 try:
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	690 if not fl.addgroup(source, revmap, trp):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	691 raise error.Abort(_("received file revlog group is empty"))
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25624 diff changeset	692 except error.CensoredBaseError as e:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	693 raise error.Abort(_("received delta base is censored: %s") % e)
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	694 revisions += len(fl) - o
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	695 files += 1
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	696 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	697 needs = needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	698 for new in xrange(o, len(fl)):
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	699 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	700 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	701 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	702 else:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	703 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	704 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	705 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	706 del needfiles[f]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	707 repo.ui.progress(_('files'), None)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	708
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	709 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	710 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	711 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	712 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	713 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	714 except error.LookupError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	715 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	716 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	717 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	718
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	719 return revisions, files
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	720
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	721 def addchangegroup(repo, source, srctype, url, emptyok=False,
25517 513c31d8652f addchangegroup: accept an expected total number of changesets as argument Pierre-Yves David <pierre-yves.david@fb.com> parents: 25505 diff changeset	722 targetphase=phases.draft, expectedtotal=None):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	723 """Add the changegroup returned by source.read() to this repo.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	724 srctype is a string like 'push', 'pull', or 'unbundle'. url is
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	725 the URL of the repo where this changegroup is coming from.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	726
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	727 Return an integer summarizing the change to this repo:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	728 - nothing changed or no source: 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	729 - more heads than before: 1+added heads (2..n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	730 - fewer heads than before: -1-removed heads (-2..-n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	731 - number of heads stays the same: 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	732 """
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	733 repo = repo.unfiltered()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	734 def csmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	735 repo.ui.debug("add changeset %s\n" % short(x))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	736 return len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	737
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	738 def revmap(x):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	739 return cl.rev(x)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	740
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	741 if not source:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	742 return 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	743
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	744 changesets = files = revisions = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	745
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	746 tr = repo.transaction("\n".join([srctype, util.hidepassword(url)]))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	747 # The transaction could have been created before and already carries source
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	748 # information. In this case we use the top level data. We overwrite the
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	749 # argument because we need to use the top level value (if they exist) in
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	750 # this function.
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	751 srctype = tr.hookargs.setdefault('source', srctype)
3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	752 url = tr.hookargs.setdefault('url', url)
23203 3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	753
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	754 # write changelog data to temp files so concurrent readers will not see
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	755 # inconsistent view
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	756 cl = repo.changelog
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	757 cl.delayupdate(tr)
3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	758 oldheads = cl.heads()
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	759 try:
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	760 repo.hook('prechangegroup', throw=True, **tr.hookargs)
22969 805c18b27f48 addchangegroup: call `prechangegroup` hook after transaction retrieval Pierre-Yves David <pierre-yves.david@fb.com> parents: 22968 diff changeset	761
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	762 trp = weakref.proxy(tr)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	763 # pull off the changeset group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	764 repo.ui.status(_("adding changesets\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	765 clstart = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	766 class prog(object):
25574 88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	767 def __init__(self, step, total):
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	768 self._step = step
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	769 self._total = total
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	770 self._count = 1
25573 3e89e67f17e8 changegroup: don't use 'repo' for non-repo 'self' Martin von Zweigbergk <martinvonz@google.com> parents: 25517 diff changeset	771 def __call__(self):
25574 88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	772 repo.ui.progress(self._step, self._count, unit=_('chunks'),
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	773 total=self._total)
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	774 self._count += 1
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	775 source.callback = prog(_('changesets'), expectedtotal)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	776
25823 2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	777 efiles = set()
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	778 def onchangelog(cl, node):
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	779 efiles.update(cl.read(node)[3])
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	780
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	781 source.changelogheader()
25823 2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	782 srccontent = cl.addgroup(source, csmap, trp,
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	783 addrevisioncb=onchangelog)
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	784 efiles = len(efiles)
2406e2baa937 changegroup: compute seen files as changesets are added (issue4750) Gregory Szorc <gregory.szorc@gmail.com> parents: 25695 diff changeset	785
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	786 if not (srccontent or emptyok):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	787 raise error.Abort(_("received changelog group is empty"))
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	788 clend = len(cl)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	789 changesets = clend - clstart
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	790 repo.ui.progress(_('changesets'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	791
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	792 # pull off the manifest group
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	793 repo.ui.status(_("adding manifests\n"))
25574 88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	794 # manifests <= changesets
88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	795 source.callback = prog(_('manifests'), changesets)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	796 # no need to check for empty manifest group here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	797 # if the result of the merge of 1 and 2 is the same in 3 and 4,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	798 # no new manifest will be created and the manifest group will
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	799 # be empty during the pull
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	800 source.manifestheader()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	801 repo.manifest.addgroup(source, revmap, trp)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	802 repo.ui.progress(_('manifests'), None)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	803
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	804 needfiles = {}
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	805 if repo.ui.configbool('server', 'validate', default=False):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	806 # validate incoming csets have their manifests
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	807 for cset in xrange(clstart, clend):
24900 59d1bf41af85 changegroup: don't reuse 'mfest' variable for different type Martin von Zweigbergk <martinvonz@google.com> parents: 24899 diff changeset	808 mfnode = repo.changelog.read(repo.changelog.node(cset))[0]
59d1bf41af85 changegroup: don't reuse 'mfest' variable for different type Martin von Zweigbergk <martinvonz@google.com> parents: 24899 diff changeset	809 mfest = repo.manifest.readdelta(mfnode)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	810 # store file nodes we must see
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	811 for f, n in mfest.iteritems():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	812 needfiles.setdefault(f, set()).add(n)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	813
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	814 # process the files
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	815 repo.ui.status(_("adding file changes\n"))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	816 source.callback = None
25574 88a17ccf0196 changegroup: simplify by not reusing 'prog(ress)' instance Martin von Zweigbergk <martinvonz@google.com> parents: 25573 diff changeset	817 pr = prog(_('files'), efiles)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	818 newrevs, newfiles = addchangegroupfiles(repo, source, revmap, trp, pr,
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	819 needfiles)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	820 revisions += newrevs
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	821 files += newfiles
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	822
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	823 dh = 0
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	824 if oldheads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	825 heads = cl.heads()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	826 dh = len(heads) - len(oldheads)
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	827 for h in heads:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	828 if h not in oldheads and repo[h].closesbranch():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	829 dh -= 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	830 htext = ""
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	831 if dh:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	832 htext = _(" (%+d heads)") % dh
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	833
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	834 repo.ui.status(_("added %d changesets"
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	835 " with %d changes to %d files%s\n")
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	836 % (changesets, revisions, files, htext))
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	837 repo.invalidatevolatilesets()
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	838
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	839 if changesets > 0:
23203 3872d563e01a changelog: handle writepending in the transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23181 diff changeset	840 p = lambda: tr.writepending() and repo.root or ""
21151 240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	841 if 'node' not in tr.hookargs:
240d340fe152 addchangegroup: register data in tr.hookargs Pierre-Yves David <pierre-yves.david@fb.com> parents: 21063 diff changeset	842 tr.hookargs['node'] = hex(cl.node(clstart))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	843 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	844 else:
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	845 hookargs = dict(tr.hookargs)
7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	846 hookargs['node'] = hex(cl.node(clstart))
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	847 repo.hook('pretxnchangegroup', throw=True, pending=p, **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	848
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	849 added = [cl.node(r) for r in xrange(clstart, clend)]
25624 f0745da75056 publishing: use new helper method Matt Mackall <mpm@selenic.com> parents: 25574 diff changeset	850 publishing = repo.publishing()
20966 63659b809021 phase: apply publishing enforcement for "serve" source Pierre-Yves David <pierre-yves.david@fb.com> parents: 20942 diff changeset	851 if srctype in ('push', 'serve'):
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	852 # Old servers can not push the boundary themselves.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	853 # New servers won't push the boundary if changeset already
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	854 # exists locally as secret
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	855 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	856 # We should not use added here but the list of all change in
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	857 # the bundle
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	858 if publishing:
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	859 phases.advanceboundary(repo, tr, phases.public, srccontent)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	860 else:
22041 97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	861 # Those changesets have been pushed from the outside, their
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	862 # phases are going to be pushed alongside. Therefor
97f86ce79abe changegroup: add a `targetphase` argument to `addchangegroup` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21260 diff changeset	863 # `targetphase` is ignored.
22069 616a455b02ca phase: add a transaction argument to advanceboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22041 diff changeset	864 phases.advanceboundary(repo, tr, phases.draft, srccontent)
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	865 phases.retractboundary(repo, tr, phases.draft, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	866 elif srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	867 # publishing only alter behavior during push
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	868 #
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	869 # strip should not touch boundary at all
22070 c1ca47204590 phase: add a transaction argument to retractboundary Pierre-Yves David <pierre-yves.david@fb.com> parents: 22069 diff changeset	870 phases.retractboundary(repo, tr, targetphase, added)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	871
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	872 if changesets > 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	873 if srctype != 'strip':
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	874 # During strip, branchcache is invalid but coming call to
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	875 # `destroyed` will repair it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	876 # In other case we can safely update cache on disk.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	877 branchmap.updatecache(repo.filtered('served'))
22960 7c13c9404c2c changegroup: use a copy of hookargs when invoking the changegroup hook Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	878
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	879 def runhooks():
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	880 # These hooks run when the lock releases, not when the
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	881 # transaction closes. So it's possible for the changelog
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	882 # to have changed since we last saw it.
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	883 if clstart >= len(repo):
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	884 return
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	885
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	886 # forcefully update the on-disk branch cache
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	887 repo.ui.debug("updating the branch cache\n")
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	888 repo.hook("changegroup", **hookargs)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	889
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	890 for n in added:
22968 01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	891 args = hookargs.copy()
01e98eba3bc1 addchangegroup: get the `node` argument of `incoming` hook from transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 22960 diff changeset	892 args['node'] = hex(n)
22971 3fe571c74b27 changegroup: store source and url in the `hookargs` dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 22970 diff changeset	893 repo.hook("incoming", **args)
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	894
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	895 newheads = [h for h in repo.heads() if h not in oldheads]
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	896 repo.ui.log("incoming",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	897 "%s incoming changes - new heads: %s\n",
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	898 len(added),
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	899 ', '.join([hex(c[:6]) for c in newheads]))
23221 cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	900
cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	901 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
23282 6c1351352b6c transaction: pass the transaction to 'postclose' callback Pierre-Yves David <pierre-yves.david@fb.com> parents: 23244 diff changeset	902 lambda tr: repo._afterlock(runhooks))
23221 cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	903
cadc9a723d60 changegroup: use the 'postclose' API on transaction Pierre-Yves David <pierre-yves.david@fb.com> parents: 23205 diff changeset	904 tr.close()
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	905
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	906 finally:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	907 tr.release()
24717 90f2b9de30f2 changegroup: flush the ui stdio buffers after adding a changegroup Matt Harbison <matt_harbison@yahoo.com> parents: 24686 diff changeset	908 repo.ui.flush()
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	909 # never return 0 here:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	910 if dh < 0:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	911 return dh - 1
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	912 else:
d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	913 return dh + 1

Mercurial > public > mercurial-scm > hg

annotate mercurial/changegroup.py @ 26595:be0489770925