mercurial-scm/hg-stable: mercurial/changegroup.py annotate

annotate mercurial/changegroup.py @ 38947:1469584ad5fe

changegroup: specify ellipses mode explicitly Currently, code throughout changegroup relies on the presence of self._full_nodes to enable ellipses mode. This is a very tenuous check. And the check may be wrong once we move _full_nodes into cgpacker. Let's capture the enabling of ellipses mode explicitly as a constructor argument and as an instance variable. We could probably derive ellipses mode by presence of other variables. But for now, this explicit approach seems simplest since it is most compatible with existing code. Differential Revision: https://phab.mercurial-scm.org/D4090

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Fri, 03 Aug 2018 14:00:18 -0700
parents	ad4c4cc9a5ac
children	1af339c22aeb

rev	line source
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9437 diff changeset	6 # GNU General Public License version 2 or any later version.
3877 abaee83ce0a6 Replace demandload with new demandimport Matt Mackall <mpm@selenic.com> parents: 3859 diff changeset	7
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	9
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	10 import os
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	11 import struct
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	12 import weakref
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	13
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	14 from .i18n import _
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	15 from .node import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	16 hex,
38922 ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38874 diff changeset	17 nullid,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	18 nullrev,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	19 short,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	20 )
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	21
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	22 from .thirdparty import (
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	23 attr,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	24 )
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	25
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	26 from . import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	27 dagutil,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	28 error,
38846 5742d0428ed9 changegroup: inline prune() logic from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38834 diff changeset	29 manifest,
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	30 match as matchmod,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	31 mdiff,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	32 phases,
30945 82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2) Pulkit Goyal <7895pulkit@gmail.com> parents: 30743 diff changeset	33 pycompat,
38874 a232e6744ba3 narrow: move requirement constant from changegroup to repository Martin von Zweigbergk <martinvonz@google.com> parents: 38846 diff changeset	34 repository,
38922 ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38874 diff changeset	35 revlog,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	36 util,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	37 )
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	38
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	39 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	40 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	41 )
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	42
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	43 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	44 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	45 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	46
37135 a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	47 LFS_REQUIREMENT = 'lfs'
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	48
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35051 diff changeset	49 readexactly = util.readexactly
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	50
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	51 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	52 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	53 d = readexactly(stream, 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	54 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	55 if l <= 4:
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	56 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	57 raise error.Abort(_("invalid chunk length %d") % l)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	58 return ""
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	59 return readexactly(stream, l - 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	60
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	61 def chunkheader(length):
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	62 """return a changegroup chunk header (string)"""
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	63 return struct.pack(">l", length + 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	64
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	65 def closechunk():
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	66 """return a changegroup chunk header (string) for a zero-length chunk"""
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	67 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	68
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	69 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	70 """Write chunks to a file and return its filename.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	71
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	72 The stream is assumed to be a bundle file.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	73 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	74 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	75 """
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	76 fh = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	77 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	78 try:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	79 if filename:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	80 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	81 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	82 else:
30212 260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	83 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	84 # small (4k is common on Linux).
260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	85 fh = open(filename, "wb", 131072)
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	86 else:
38197 aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path Yuya Nishihara <yuya@tcha.org> parents: 37339 diff changeset	87 fd, filename = pycompat.mkstemp(prefix="hg-bundle-", suffix=".hg")
36843 5bc7ff103081 py3: use r'' instead of sysstr('') to get around code transformer Yuya Nishihara <yuya@tcha.org> parents: 36768 diff changeset	88 fh = os.fdopen(fd, r"wb")
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	89 cleanup = filename
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	90 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	91 fh.write(c)
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	92 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	93 return filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	94 finally:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	95 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	96 fh.close()
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	97 if cleanup is not None:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	98 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	99 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	100 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	101 os.unlink(cleanup)
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	102
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	103 class cg1unpacker(object):
26708 749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	104 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	105
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	106 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	107 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	108 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	109
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	110 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	111 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	112 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	113 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	114
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	115 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	116 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	117
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	118 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	119 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	120 """
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	121 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	122 deltaheadersize = deltaheader.size
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	123 version = '01'
27920 da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	124 _grouplistcount = 1 # One list of files after the manifests
da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	125
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	126 def __init__(self, fh, alg, extras=None):
30364 a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	127 if alg is None:
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	128 alg = 'UN'
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	129 if alg not in util.compengines.supportedbundletypes:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	130 raise error.Abort(_('unknown stream compression type: %s')
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	131 % alg)
26392 127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	132 if alg == 'BZ':
127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	133 alg = '_truncatedBZ'
30364 a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	134
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	135 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	136 self._stream = compengine.decompressorreader(fh)
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	137 self._type = alg
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	138 self.extras = extras or {}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	139 self.callback = None
26706 8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	140
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	141 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	142 # be used by bundlerepo, but it's a little hard to tell.
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	143 def compressed(self):
30594 182cacaa4c32 cg1packer: fix `compressed` method Stanislau Hlebik <stash@fb.com> parents: 30364 diff changeset	144 return self._type is not None and self._type != 'UN'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	145 def read(self, l):
bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	146 return self._stream.read(l)
12330 e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	147 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	148 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	149 def tell(self):
12332 680fe77ab5b8 bundlerepo: use bundle objects everywhere Matt Mackall <mpm@selenic.com> parents: 12330 diff changeset	150 return self._stream.tell()
12347 6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	151 def close(self):
6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	152 return self._stream.close()
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	153
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	154 def _chunklength(self):
13459 acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34 Jim Hague <jim.hague@acm.org> parents: 13458 diff changeset	155 d = readexactly(self._stream, 4)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	156 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	157 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	158 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	159 raise error.Abort(_("invalid chunk length %d") % l)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	160 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	161 if self.callback:
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	162 self.callback()
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	163 return l - 4
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	164
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	165 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	166 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	167 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	168
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	169 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	170 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	171 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	172
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	173 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	174 """return the header of the filelogs chunk, v10 only has the filename"""
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	175 l = self._chunklength()
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	176 if not l:
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	177 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	178 fname = readexactly(self._stream, l)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	179 return {'filename': fname}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	180
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	181 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	182 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	183 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	184 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	185 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	186 deltabase = prevnode
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	187 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	188 return node, p1, p2, deltabase, cs, flags
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	189
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	190 def deltachunk(self, prevnode):
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	191 l = self._chunklength()
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	192 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	193 return {}
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	194 headerdata = readexactly(self._stream, self.deltaheadersize)
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	195 header = self.deltaheader.unpack(headerdata)
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	196 delta = readexactly(self._stream, l - self.deltaheadersize)
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	197 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
34301 05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	198 return (node, p1, p2, cs, deltabase, delta, flags)
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	199
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	200 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	201 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	202
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	203 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	204 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	205 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	206 """
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	207 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	208 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	209 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	210 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	211 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	212 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	213 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	214 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	215 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	216 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	217 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	218
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	219 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	220 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	221 noentries = True
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	222 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	223 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	224 if not chunk:
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	225 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	226 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	227 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	228 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	229 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	230 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	231 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	232 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	233 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	234 parts += 1
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	235 break
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	236 noentries = False
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	237 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	238 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	239 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	240 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	241 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	242 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	243 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	244
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	245 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	246 self.callback = prog.increment
26712 04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	247 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	248 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	249 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	250 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	251 self.manifestheader()
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	252 deltas = self.deltaiter()
38557 0db41eb0a3ac manifest: define and implement addgroup() on manifestlog Gregory Szorc <gregory.szorc@gmail.com> parents: 38416 diff changeset	253 repo.manifestlog.addgroup(deltas, revmap, trp)
38379 ef692614e601 progress: hide update(None) in a new complete() method Martin von Zweigbergk <martinvonz@google.com> parents: 38352 diff changeset	254 prog.complete()
28360 11287888ce4b changegroup: exclude submanifests from manifest progress Martin von Zweigbergk <martinvonz@google.com> parents: 28281 diff changeset	255 self.callback = None
26712 04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	256
33308 248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	257 def apply(self, repo, tr, srctype, url, targetphase=phases.draft,
248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	258 expectedtotal=None):
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	259 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	260 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	261 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	262
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	263 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	264 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	265 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	266 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	267 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	268 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	269 repo = repo.unfiltered()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	270 def csmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	271 repo.ui.debug("add changeset %s\n" % short(x))
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	272 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	273
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	274 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	275 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	276
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	277 changesets = files = revisions = 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	278
26880 fa7f8b686633 changegroup: fix the scope of a try finally Pierre-Yves David <pierre-yves.david@fb.com> parents: 26859 diff changeset	279 try:
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	280 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	281 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	282 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	283 # in this function.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	284 srctype = tr.hookargs.setdefault('source', srctype)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	285 url = tr.hookargs.setdefault('url', url)
33697 38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	286 repo.hook('prechangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	287 throw=True, **pycompat.strkwargs(tr.hookargs))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	288
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	289 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	290 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	291 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	292 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	293 oldheads = set(cl.heads())
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	294
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	295 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	296 # pull off the changeset group
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	297 repo.ui.status(_("adding changesets\n"))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	298 clstart = len(cl)
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	299 progress = repo.ui.makeprogress(_('changesets'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	300 total=expectedtotal)
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	301 self.callback = progress.increment
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	302
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	303 efiles = set()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	304 def onchangelog(cl, node):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	305 efiles.update(cl.readfiles(node))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	306
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	307 self.changelogheader()
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	308 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	309 cgnodes = cl.addgroup(deltas, csmap, trp, addrevisioncb=onchangelog)
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	310 efiles = len(efiles)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	311
33308 248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	312 if not cgnodes:
33309 69c4493a54f9 changegroup: don't fail on empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33308 diff changeset	313 repo.ui.develwarn('applied empty changegroup',
34734 3572b2031cec devel-warn: add 'warn-' to 'devel.empty-changegroup' config Boris Feld <boris.feld@octobus.net> parents: 34301 diff changeset	314 config='warn-empty-changegroup')
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	315 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	316 changesets = clend - clstart
38379 ef692614e601 progress: hide update(None) in a new complete() method Martin von Zweigbergk <martinvonz@google.com> parents: 38352 diff changeset	317 progress.complete()
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	318 self.callback = None
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	319
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	320 # pull off the manifest group
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	321 repo.ui.status(_("adding manifests\n"))
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	322 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	323 # changesets.
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	324 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	325 total=changesets)
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	326 self._unpackmanifests(repo, revmap, trp, progress)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	327
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	328 needfiles = {}
33227 86c9aa1d598f configitems: register the 'server.validate' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33185 diff changeset	329 if repo.ui.configbool('server', 'validate'):
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	330 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	331 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	332 # validate incoming csets have their manifests
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	333 for cset in pycompat.xrange(clstart, clend):
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	334 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	335 mfest = ml[mfnode].readdelta()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	336 # store file cgnodes we must see
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	337 for f, n in mfest.iteritems():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	338 needfiles.setdefault(f, set()).add(n)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	339
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	340 # process the files
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	341 repo.ui.status(_("adding file changes\n"))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	342 newrevs, newfiles = _addchangegroupfiles(
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	343 repo, self, revmap, trp, efiles, needfiles)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	344 revisions += newrevs
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	345 files += newfiles
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	346
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	347 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	348 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	349 heads = cl.heads()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	350 deltaheads = len(heads) - len(oldheads)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	351 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	352 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	353 deltaheads -= 1
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	354 htext = ""
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	355 if deltaheads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	356 htext = _(" (%+d heads)") % deltaheads
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	357
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	358 repo.ui.status(_("added %d changesets"
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	359 " with %d changes to %d files%s\n")
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	360 % (changesets, revisions, files, htext))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	361 repo.invalidatevolatilesets()
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	362
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	363 if changesets > 0:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	364 if 'node' not in tr.hookargs:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	365 tr.hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	366 tr.hookargs['node_last'] = hex(cl.node(clend - 1))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	367 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	368 else:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	369 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	370 hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	371 hookargs['node_last'] = hex(cl.node(clend - 1))
33697 38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	372 repo.hook('pretxnchangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	373 throw=True, **pycompat.strkwargs(hookargs))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	374
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	375 added = [cl.node(r) for r in pycompat.xrange(clstart, clend)]
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	376 phaseall = None
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	377 if srctype in ('push', 'serve'):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	378 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	379 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	380 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	381 #
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	382 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	383 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	384 if repo.publishing():
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	385 targetphase = phaseall = phases.public
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	386 else:
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	387 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	388
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	389 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	390 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	391 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	392 # ignored.
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	393 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	394 if added:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	395 phases.registernew(repo, tr, targetphase, added)
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	396 if phaseall is not None:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	397 phases.advanceboundary(repo, tr, phaseall, cgnodes)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	398
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	399 if changesets > 0:
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	400
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	401 def runhooks():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	402 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	403 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	404 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	405 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	406 return
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	407
33738 672ad4f3bb84 changegroup: more *kwargs Augie Fackler <augie@google.com>* parents: 33697 diff changeset	408 repo.hook("changegroup", **pycompat.strkwargs(hookargs))
27867 7ced54ebf972 with: use context manager for transaction in changegroup apply Bryan O'Sullivan <bryano@fb.com> parents: 27754 diff changeset	409
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	410 for n in added:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	411 args = hookargs.copy()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	412 args['node'] = hex(n)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	413 del args['node_last']
33738 672ad4f3bb84 changegroup: more *kwargs Augie Fackler <augie@google.com>* parents: 33697 diff changeset	414 repo.hook("incoming", **pycompat.strkwargs(args))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	415
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	416 newheads = [h for h in repo.heads()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	417 if h not in oldheads]
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	418 repo.ui.log("incoming",
36760 2a5024109490 py3: fix int formatting of "incoming changes" log Yuya Nishihara <yuya@tcha.org> parents: 36495 diff changeset	419 "%d incoming changes - new heads: %s\n",
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	420 len(added),
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	421 ', '.join([hex(c[:6]) for c in newheads]))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	422
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	423 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	424 lambda tr: repo._afterlock(runhooks))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	425 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	426 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	427 # never return 0 here:
32888 b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads" Martin von Zweigbergk <martinvonz@google.com> parents: 32887 diff changeset	428 if deltaheads < 0:
33042 3e102a8dd52c bundle2: record changegroup data in 'op.records' (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32949 diff changeset	429 ret = deltaheads - 1
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	430 else:
33042 3e102a8dd52c bundle2: record changegroup data in 'op.records' (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32949 diff changeset	431 ret = deltaheads + 1
33461 bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply' Boris Feld <boris.feld@octobus.net> parents: 33456 diff changeset	432 return ret
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	433
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	434 def deltaiter(self):
34160 c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	435 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	436 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	437
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	438 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	439 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	440 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	441 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34301 05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	442 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	443 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	444 chain = chunkdata[0]
34160 c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	445
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	446 class cg2unpacker(cg1unpacker):
26708 749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	447 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	448
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	449 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	450 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	451 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	452 """
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	453 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	454 deltaheadersize = deltaheader.size
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	455 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	456
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	457 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	458 node, p1, p2, deltabase, cs = headertuple
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	459 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	460 return node, p1, p2, deltabase, cs, flags
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	461
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	462 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	463 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	464
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	465 cg3 streams add support for exchanging treemanifests and revlog
27753 d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	466 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	467 separating manifests and files.
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	468 """
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	469 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	470 deltaheadersize = deltaheader.size
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	471 version = '03'
27920 da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	472 _grouplistcount = 2 # One list of manifests and one list of files
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	473
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	474 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	475 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	476 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	477
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	478 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	479 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29736 4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29704 diff changeset	480 for chunkdata in iter(self.filelogheader, {}):
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	481 # If we get here, there are directory manifests in the changegroup
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	482 d = chunkdata["filename"]
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	483 repo.ui.debug("adding %s revisions\n" % d)
30349 6cdfb7e15a35 changegroup: remove remaining uses of repo.manifest Durham Goode <durham@fb.com> parents: 30308 diff changeset	484 dirlog = repo.manifestlog._revlog.dirlog(d)
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	485 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	486 if not dirlog.addgroup(deltas, revmap, trp):
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	487 raise error.Abort(_("received dir revlog group is empty"))
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	488
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	489 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	490 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	491 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	492 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	493 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	494 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	495 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	496 if len(d) < n:
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	497 d += readexactly(self._fh, n - len(d))
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	498 return d
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	499 return readexactly(self._fh, n)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	500
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	501 @attr.s(slots=True, frozen=True)
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	502 class revisiondelta(object):
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	503 """Describes a delta entry in a changegroup.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	504
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	505 Captured data is sufficient to serialize the delta into multiple
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	506 formats.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	507 """
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	508 # 20 byte node of this revision.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	509 node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	510 # 20 byte nodes of parent revisions.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	511 p1node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	512 p2node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	513 # 20 byte node of node this delta is against.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	514 basenode = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	515 # 20 byte node of changeset revision this delta is associated with.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	516 linknode = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	517 # 2 bytes of flags to apply to revision data.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	518 flags = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	519 # Iterable of chunks holding raw delta data.
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	520 deltachunks = attr.ib()
38922 ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38874 diff changeset	521
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	522 class cgpacker(object):
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	523 def __init__(self, repo, filematcher, version, allowreorder,
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	524 useprevdelta, builddeltaheader, manifestsend,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	525 sendtreemanifests, bundlecaps=None, ellipses=False,
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	526 shallow=False, ellipsisroots=None):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	527 """Given a source repo, construct a bundler.
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	528
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	529 filematcher is a matcher that matches on files to include in the
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	530 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	531
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	532 allowreorder controls whether reordering of revisions is allowed.
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	533 This value is used when ``bundle.reorder`` is ``auto`` or isn't
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	534 set.
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	535
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	536 useprevdelta controls whether revisions should always delta against
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	537 the previous revision in the changegroup.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	538
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	539 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	540 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	541
38937 67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	542 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	543
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	544 sendtreemanifests indicates whether tree manifests should be emitted.
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	545
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	546 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	547
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	548 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	549 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	550 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	551 capabilities to customize the changegroup packer.
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	552
cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	553 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	554 need to pack file contents not introduced by the changes being packed.
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	555 """
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	556 assert filematcher
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	557 self._filematcher = filematcher
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	558
38934 d7ac49c2353c changegroup: pass version into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38933 diff changeset	559 self.version = version
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	560 self._useprevdelta = useprevdelta
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	561 self._builddeltaheader = builddeltaheader
38937 67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	562 self._manifestsend = manifestsend
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	563 self._sendtreemanifests = sendtreemanifests
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	564 self._ellipses = ellipses
38934 d7ac49c2353c changegroup: pass version into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38933 diff changeset	565
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	566 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	567 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	568 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	569 self._bundlecaps = bundlecaps
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	570 self._isshallow = shallow
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	571
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	572 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	573 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	574
25831 578fc97904da generaldelta: mark experimental reordering option Matt Mackall <mpm@selenic.com> parents: 25823 diff changeset	575 # experimental config: bundle.reorder
33185 9f95f0bb343b configitems: register the 'bundle.reorder' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33048 diff changeset	576 reorder = repo.ui.config('bundle', 'reorder')
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	577 if reorder == 'auto':
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	578 self._reorder = allowreorder
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	579 else:
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	580 self._reorder = stringutil.parsebool(reorder)
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	581
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	582 self._repo = repo
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	583
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	584 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	585 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	586 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	587 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	588
38944 379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	589 # TODO the functionality keyed off of this should probably be
379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	590 # controlled via arguments to group() that influence behavior.
379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	591 self._changelogdone = False
379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	592
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	593 # Maps CL revs to per-revlog revisions. Cleared in close() at
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	594 # the end of each group.
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	595 self._clrevtolocalrev = {}
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	596 self._nextclrevtolocalrev = {}
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	597
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	598 # Maps changelog nodes to changelog revs. Filled in once
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	599 # during changelog stage and then left unmodified.
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	600 self._clnodetorev = {}
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	601
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	602 def _close(self):
38926 75d6139e69f9 changegroup: move close() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38925 diff changeset	603 # Ellipses serving mode.
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	604 self._clrevtolocalrev.clear()
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	605 if self._nextclrevtolocalrev:
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	606 self.clrevtolocalrev = self._nextclrevtolocalrev
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	607 self._nextclrevtolocalrev.clear()
38944 379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	608 self._changelogdone = True
38926 75d6139e69f9 changegroup: move close() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38925 diff changeset	609
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	610 return closechunk()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	611
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	612 def _fileheader(self, fname):
13831 d69c9510d648 changegroup: introduce bundler objects Matt Mackall <mpm@selenic.com> parents: 13786 diff changeset	613 return chunkheader(len(fname)) + fname
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	614
29236 1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	615 # Extracted both for clarity and for overriding in extensions.
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	616 def _sortgroup(self, store, nodelist, lookup):
29236 1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	617 """Sort nodes for change group and turn them into revnums."""
38927 c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	618 # Ellipses serving mode.
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	619 #
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	620 # In a perfect world, we'd generate better ellipsis-ified graphs
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	621 # for non-changelog revlogs. In practice, we haven't started doing
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	622 # that yet, so the resulting DAGs for the manifestlog and filelogs
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	623 # are actually full of bogus parentage on all the ellipsis
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	624 # nodes. This has the side effect that, while the contents are
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	625 # correct, the individual DAGs might be completely out of whack in
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	626 # a case like 882681bc3166 and its ancestors (back about 10
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	627 # revisions or so) in the main hg repo.
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	628 #
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	629 # The one invariant we know holds is that the new (potentially
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	630 # bogus) DAG shape will be valid if we order the nodes in the
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	631 # order that they're introduced in dramatis personae by the
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	632 # changelog, so what we do is we sort the non-changelog histories
c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	633 # by the order in which they are used by the changelog.
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	634 if self._ellipses and self._clnodetorev:
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	635 key = lambda n: self._clnodetorev[lookup(n)]
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	636 return [store.rev(n) for n in sorted(nodelist, key=key)]
38927 c9315bc578bc changegroup: move _sortgroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38926 diff changeset	637
29236 1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	638 # for generaldelta revlogs, we linearize the revs; this will both be
1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	639 # much quicker and generate a much smaller bundle
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	640 if (store._generaldelta and self._reorder is None) or self._reorder:
205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	641 dag = dagutil.revlogdag(store)
205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	642 return dag.linearize(set(store.rev(n) for n in nodelist))
29236 1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	643 else:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	644 return sorted([store.rev(n) for n in nodelist])
29236 1b7d907ec18a changegroup: extract method that sorts nodes to send Augie Fackler <augie@google.com> parents: 28666 diff changeset	645
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	646 def group(self, nodelist, store, lookup, units=None):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	647 """Calculate a delta group, yielding a sequence of changegroup chunks
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	648 (strings).
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	649
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	650 Given a list of changeset revs, return a set of deltas and
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	651 metadata corresponding to nodes. The first delta is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	652 first parent(nodelist[0]) -> nodelist[0], the receiver is
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	653 guaranteed to have this parent as it has all history before
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	654 these changesets. In the case firstparent is nullrev the
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	655 changegroup starts with a full revision.
19208 0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	656
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	657 If units is not None, progress detail will be generated, units specifies
0b564cf359a7 bundle-ng: move progress handling out of the linkrev callback Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19207 diff changeset	658 the type of revlog that is touched (changelog, manifest, etc.).
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	659 """
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	660 # if we don't have any revisions touched by these changesets, bail
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	661 if len(nodelist) == 0:
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	662 yield self._close()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	663 return
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	664
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	665 revs = self._sortgroup(store, nodelist, lookup)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	666
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	667 # add the parent of the first rev
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	668 p = store.parentrevs(revs[0])[0]
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	669 revs.insert(0, p)
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	670
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	671 # build deltas
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	672 progress = None
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	673 if units is not None:
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	674 progress = self._repo.ui.makeprogress(_('bundling'), unit=units,
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	675 total=(len(revs) - 1))
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	676 for r in pycompat.xrange(len(revs) - 1):
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	677 if progress:
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	678 progress.update(r + 1)
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	679 prev, curr = revs[r], revs[r + 1]
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	680 linknode = lookup(store.node(curr))
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	681 for c in self._revchunk(store, curr, prev, linknode):
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	682 yield c
4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	683
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	684 if progress:
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	685 progress.complete()
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	686 yield self._close()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	687
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	688 # filter any nodes that claim to be part of the known set
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	689 def _prune(self, store, missing, commonrevs):
38846 5742d0428ed9 changegroup: inline prune() logic from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38834 diff changeset	690 # TODO this violates storage abstraction for manifests.
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	691 if isinstance(store, manifest.manifestrevlog):
205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	692 if not self._filematcher.visitdir(store._dir[:-1] or '.'):
38846 5742d0428ed9 changegroup: inline prune() logic from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38834 diff changeset	693 return []
5742d0428ed9 changegroup: inline prune() logic from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38834 diff changeset	694
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	695 rr, rl = store.rev, store.linkrev
19289 6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	696 return [n for n in missing if rl(rr(n)) not in commonrevs]
6ea1f858efd9 bundle: refactor changegroup prune to be its own function Durham Goode <durham@fb.com> parents: 19208 diff changeset	697
28228 abf120262683 changegroup: make _packmanifests() dumber Martin von Zweigbergk <martinvonz@google.com> parents: 28227 diff changeset	698 def _packmanifests(self, dir, mfnodes, lookuplinknode):
26711 0ef0aec56090 changegroup: move manifest packing into a separate function Augie Fackler <augie@google.com> parents: 26710 diff changeset	699 """Pack flat manifests into a changegroup stream."""
28228 abf120262683 changegroup: make _packmanifests() dumber Martin von Zweigbergk <martinvonz@google.com> parents: 28227 diff changeset	700 assert not dir
30349 6cdfb7e15a35 changegroup: remove remaining uses of repo.manifest Durham Goode <durham@fb.com> parents: 30308 diff changeset	701 for chunk in self.group(mfnodes, self._repo.manifestlog._revlog,
28228 abf120262683 changegroup: make _packmanifests() dumber Martin von Zweigbergk <martinvonz@google.com> parents: 28227 diff changeset	702 lookuplinknode, units=_('manifests')):
26711 0ef0aec56090 changegroup: move manifest packing into a separate function Augie Fackler <augie@google.com> parents: 26710 diff changeset	703 yield chunk
28228 abf120262683 changegroup: make _packmanifests() dumber Martin von Zweigbergk <martinvonz@google.com> parents: 28227 diff changeset	704
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	705 def _packtreemanifests(self, dir, mfnodes, lookuplinknode):
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	706 """Version of _packmanifests that operates on directory manifests.
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	707
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	708 Encodes the directory name in the output so multiple manifests
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	709 can be sent.
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	710 """
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	711 assert self.version == b'03'
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	712
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	713 if dir:
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	714 yield self._fileheader(dir)
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	715
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	716 # TODO violates storage abstractions by assuming revlogs.
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	717 dirlog = self._repo.manifestlog._revlog.dirlog(dir)
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	718 for chunk in self.group(mfnodes, dirlog, lookuplinknode,
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	719 units=_('manifests')):
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	720 yield chunk
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	721
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	722 def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	723 '''yield a sequence of changegroup chunks (strings)'''
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	724 repo = self._repo
24978 f52560c64953 changegroup: drop _changelog and _manifest properties Martin von Zweigbergk <martinvonz@google.com> parents: 24977 diff changeset	725 cl = repo.changelog
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	726
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	727 clrevorder = {}
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	728 mfs = {} # needed manifests
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	729 fnodes = {} # needed file nodes
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	730 mfl = repo.manifestlog
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	731 # TODO violates storage abstraction.
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	732 mfrevlog = mfl._revlog
28241 a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	733 changedfiles = set()
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	734
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	735 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	736 # manifest nodes.
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	737 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	738 def lookupcl(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	739 c = cl.read(x)
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	740 clrevorder[x] = len(clrevorder)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	741
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	742 if self._ellipses:
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	743 # Only update mfs if x is going to be sent. Otherwise we
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	744 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	745 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	746 # have sent.
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	747 if (x in self._full_nodes
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	748 or cl.rev(x) in self._precomputedellipsis):
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	749 n = c[0]
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	750 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	751 # version.
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	752 mfs.setdefault(n, x)
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	753 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	754 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	755 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	756 # parents)
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	757 self._nextclrevtolocalrev.setdefault(cl.rev(x),
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	758 mfrevlog.rev(n))
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	759 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	760 # client requested a shallow clone.
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	761 if self._isshallow:
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	762 changedfiles.update(mfl[c[0]].read().keys())
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	763 else:
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	764 changedfiles.update(c[3])
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	765 else:
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	766
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	767 n = c[0]
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	768 # record the first changeset introducing this manifest version
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	769 mfs.setdefault(n, x)
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	770 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	771 # this manifest.
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	772 changedfiles.update(c[3])
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	773
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	774 return x
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	775
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	776 self._verbosenote(_('uncompressed size of bundle content:\n'))
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	777 size = 0
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	778 for chunk in self.group(clnodes, cl, lookupcl, units=_('changesets')):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	779 size += len(chunk)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	780 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	781 self._verbosenote(_('%8.i (changelog)\n') % size)
23224 f4ab47ccefde changegroup: don't define lookupmf() until it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 22971 diff changeset	782
24977 4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	783 # We need to make sure that the linkrev in the changegroup refers to
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	784 # the first changeset that introduced the manifest or file revision.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	785 # The fastpath is usually safer than the slowpath, because the filelogs
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	786 # are walked in revlog order.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	787 #
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	788 # When taking the slowpath with reorder=None and the manifest revlog
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	789 # uses generaldelta, the manifest may be walked in the "wrong" order.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	790 # Without 'clrevorder', we would get an incorrect linkrev (see fix in
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	791 # cc0ff93d0c0c).
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	792 #
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	793 # When taking the fastpath, we are only vulnerable to reordering
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	794 # of the changelog itself. The changelog never uses generaldelta, so
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	795 # it is only reordered when reorder=True. To handle this case, we
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	796 # simply take the slowpath, which already has the 'clrevorder' logic.
4289383cb9d2 changegroup: document the cases where reordering complicates linkrevs Martin von Zweigbergk <martinvonz@google.com> parents: 24976 diff changeset	797 # This was also fixed in cc0ff93d0c0c.
24976 147d8892fc4b changegroup: extract condition for linkrev fastpath Martin von Zweigbergk <martinvonz@google.com> parents: 24912 diff changeset	798 fastpathlinkrev = fastpathlinkrev and not self._reorder
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	799 # Treemanifests don't work correctly with fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	800 # either, because we don't discover which directory nodes to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	801 # send along with files. This could probably be fixed.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	802 fastpathlinkrev = fastpathlinkrev and (
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	803 'treemanifest' not in repo.requirements)
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	804
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	805 for chunk in self.generatemanifests(commonrevs, clrevorder,
34161 75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	806 fastpathlinkrev, mfs, fnodes, source):
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	807 yield chunk
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	808
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	809 if self._ellipses:
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	810 mfdicts = None
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	811 if self._isshallow:
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	812 mfdicts = [(self._repo.manifestlog[n].read(), lr)
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	813 for (n, lr) in mfs.iteritems()]
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	814
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	815 mfs.clear()
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	816 clrevs = set(cl.rev(x) for x in clnodes)
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	817
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	818 if not fastpathlinkrev:
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	819 def linknodes(unused, fname):
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	820 return fnodes.get(fname, {})
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	821 else:
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	822 cln = cl.node
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	823 def linknodes(filerevlog, fname):
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	824 llr = filerevlog.linkrev
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	825 fln = filerevlog.node
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	826 revs = ((r, llr(r)) for r in filerevlog)
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	827 return dict((fln(r), cln(lr)) for r, lr in revs if lr in clrevs)
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	828
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	829 if self._ellipses:
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	830 # We need to pass the mfdicts variable down into
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	831 # generatefiles(), but more than one command might have
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	832 # wrapped generatefiles so we can't modify the function
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	833 # signature. Instead, we pass the data to ourselves using an
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	834 # instance attribute. I'm sorry.
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	835 self._mfdicts = mfdicts
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	836
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	837 for chunk in self.generatefiles(changedfiles, linknodes, commonrevs,
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	838 source):
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	839 yield chunk
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	840
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	841 yield self._close()
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	842
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	843 if clnodes:
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	844 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	845
1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	846 def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, mfs,
34161 75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	847 fnodes, source):
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	848 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	849
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	850 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	851 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	852 """
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	853 repo = self._repo
30308 bce79dfcf5e4 manifest: get rid of manifest.readshallowfast Durham Goode <durham@fb.com> parents: 30282 diff changeset	854 mfl = repo.manifestlog
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast Durham Goode <durham@fb.com> parents: 30282 diff changeset	855 dirlog = mfl._revlog.dirlog
28232 829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop Martin von Zweigbergk <martinvonz@google.com> parents: 28231 diff changeset	856 tmfnodes = {'': mfs}
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	857
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	858 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	859 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	860 # Returns the linkrev node (collected in lookupcl).
35051 d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests Kyle Lippincott <spectral@google.com> parents: 34734 diff changeset	861 def makelookupmflinknode(dir, nodes):
28231 3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	862 if fastpathlinkrev:
3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	863 assert not dir
3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	864 return mfs.__getitem__
3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	865
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	866 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	867 """Callback for looking up the linknode for manifests.
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	868
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	869 Returns the linkrev node for the specified manifest.
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	870
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	871 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	872
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	873 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	874 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	875 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	876 to send
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	877
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	878 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	879 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	880 treemanifests to send.
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	881 """
35051 d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests Kyle Lippincott <spectral@google.com> parents: 34734 diff changeset	882 clnode = nodes[x]
30308 bce79dfcf5e4 manifest: get rid of manifest.readshallowfast Durham Goode <durham@fb.com> parents: 30282 diff changeset	883 mdata = mfl.get(dir, x).readfast(shallow=True)
28241 a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	884 for p, n, fl in mdata.iterentries():
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	885 if fl == 't': # subdirectory manifest
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	886 subdir = dir + p + '/'
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	887 tmfclnodes = tmfnodes.setdefault(subdir, {})
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	888 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	889 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	890 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	891 else:
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	892 f = dir + p
28240 1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	893 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	894 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	895 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	896 fclnodes[n] = clnode
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	897 return clnode
28231 3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	898 return lookupmflinknode
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	899
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	900 fn = (self._packtreemanifests if self._sendtreemanifests
7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	901 else self._packmanifests)
28228 abf120262683 changegroup: make _packmanifests() dumber Martin von Zweigbergk <martinvonz@google.com> parents: 28227 diff changeset	902 size = 0
28232 829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop Martin von Zweigbergk <martinvonz@google.com> parents: 28231 diff changeset	903 while tmfnodes:
35051 d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests Kyle Lippincott <spectral@google.com> parents: 34734 diff changeset	904 dir, nodes = tmfnodes.popitem()
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	905 prunednodes = self._prune(dirlog(dir), nodes, commonrevs)
29371 1b699c7eb2b7 changegroup: don't send empty subdirectory manifest groups Martin von Zweigbergk <martinvonz@google.com> parents: 29236 diff changeset	906 if not dir or prunednodes:
38938 7a154778fb46 changegroup: consolidate tree manifests sending into cg1packer Gregory Szorc <gregory.szorc@gmail.com> parents: 38937 diff changeset	907 for x in fn(dir, prunednodes, makelookupmflinknode(dir, nodes)):
29371 1b699c7eb2b7 changegroup: don't send empty subdirectory manifest groups Martin von Zweigbergk <martinvonz@google.com> parents: 29236 diff changeset	908 size += len(x)
1b699c7eb2b7 changegroup: don't send empty subdirectory manifest groups Martin von Zweigbergk <martinvonz@google.com> parents: 29236 diff changeset	909 yield x
28229 8e13b2379407 changegroup: include subdirectory manifests in verbose size Martin von Zweigbergk <martinvonz@google.com> parents: 28228 diff changeset	910 self._verbosenote(_('%8.i (manifests)\n') % size)
38937 67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	911 yield self._manifestsend
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	912
24897 5c35a6040352 changegroup: document that 'source' parameter exists for extensions Martin von Zweigbergk <martinvonz@google.com> parents: 24896 diff changeset	913 # The 'source' parameter is useful for extensions
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	914 def generatefiles(self, changedfiles, linknodes, commonrevs, source):
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	915 changedfiles = list(filter(self._filematcher, changedfiles))
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	916
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	917 if self._isshallow:
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	918 # See comment in generate() for why this sadness is a thing.
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	919 mfdicts = self._mfdicts
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	920 del self._mfdicts
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	921 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	922 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	923 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	924 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	925 oldlinknodes = linknodes
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	926 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	927
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	928 # Defining this function has a side-effect of overriding the
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	929 # function of the same name that was passed in as an argument.
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	930 # TODO have caller pass in appropriate function.
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	931 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	932 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	933 try:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	934 fnode = c.filenode(fname)
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	935 self._clrevtolocalrev[c.rev()] = flog.rev(fnode)
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	936 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	937 pass
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	938 links = oldlinknodes(flog, fname)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	939 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	940 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	941 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	942 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	943 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	944 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	945 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	946 return links
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	947
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	948 return self._generatefiles(changedfiles, linknodes, commonrevs, source)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	949
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	950 def _generatefiles(self, changedfiles, linknodes, commonrevs, source):
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	951 repo = self._repo
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	952 progress = repo.ui.makeprogress(_('bundling'), unit=_('files'),
1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	953 total=len(changedfiles))
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	954 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	955 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	956 if not filerevlog:
37339 5859800edfc5 changegroup: remove "revlog" from error message Gregory Szorc <gregory.szorc@gmail.com> parents: 37135 diff changeset	957 raise error.Abort(_("empty or missing file data for %s") %
5859800edfc5 changegroup: remove "revlog" from error message Gregory Szorc <gregory.szorc@gmail.com> parents: 37135 diff changeset	958 fname)
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	959
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	960 linkrevnodes = linknodes(filerevlog, fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	961 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	962 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	963 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	964 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	965
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	966 filenodes = self._prune(filerevlog, linkrevnodes, commonrevs)
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	967 if filenodes:
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	968 progress.update(i + 1, item=fname)
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	969 h = self._fileheader(fname)
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	970 size = len(h)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	971 yield h
24912 e285b98c65cc changegroup.group: drop 'reorder' parameter Martin von Zweigbergk <martinvonz@google.com> parents: 24911 diff changeset	972 for chunk in self.group(filenodes, filerevlog, lookupfilelog):
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	973 size += len(chunk)
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	974 yield chunk
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	975 self._verbosenote(_('%8.i %s\n') % (size, fname))
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	976 progress.complete()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	977
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	978 def _deltaparent(self, store, rev, p1, p2, prev):
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	979 if self._useprevdelta:
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	980 if not store.candelta(prev, rev):
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	981 raise error.ProgrammingError(
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	982 'cg1 should not be used in this case')
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	983 return prev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	984
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	985 # Narrow ellipses mode.
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	986 if self._ellipses:
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	987 # TODO: send better deltas when in narrow mode.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	988 #
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	989 # changegroup.group() loops over revisions to send,
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	990 # including revisions we'll skip. What this means is that
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	991 # `prev` will be a potentially useless delta base for all
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	992 # ellipsis nodes, as the client likely won't have it. In
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	993 # the future we should do bookkeeping about which nodes
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	994 # have been sent to the client, and try to be
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	995 # significantly smarter about delta bases. This is
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	996 # slightly tricky because this same code has to work for
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	997 # all revlogs, and we don't have the linkrev/linknode here.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	998 return p1
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	999
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1000 dp = store.deltaparent(rev)
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1001 if dp == nullrev and store.storedeltachains:
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1002 # Avoid sending full revisions when delta parent is null. Pick prev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1003 # in that case. It's tempting to pick p1 in this case, as p1 will
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1004 # be smaller in the common case. However, computing a delta against
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1005 # p1 may require resolving the raw text of p1, which could be
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1006 # expensive. The revlog caches should have prev cached, meaning
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1007 # less CPU for changegroup generation. There is likely room to add
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1008 # a flag and/or config option to control this behavior.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1009 base = prev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1010 elif dp == nullrev:
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1011 # revlog is configured to use full snapshot for a reason,
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1012 # stick to full snapshot.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1013 base = nullrev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1014 elif dp not in (p1, p2, prev):
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1015 # Pick prev when we can't be sure remote has the base revision.
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1016 return prev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1017 else:
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1018 base = dp
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1019
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1020 if base != nullrev and not store.candelta(base, rev):
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1021 base = nullrev
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1022
23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	1023 return base
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	1024
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	1025 def _revchunk(self, store, rev, prev, linknode):
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1026 if self._ellipses:
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1027 fn = self._revisiondeltanarrow
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1028 else:
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1029 fn = self._revisiondeltanormal
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1030
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1031 delta = fn(store, rev, prev, linknode)
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1032 if not delta:
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1033 return
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1034
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1035 meta = self._builddeltaheader(delta)
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1036 l = len(meta) + sum(len(x) for x in delta.deltachunks)
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1037
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1038 yield chunkheader(l)
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1039 yield meta
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1040 for x in delta.deltachunks:
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1041 yield x
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1042
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1043 def _revisiondeltanormal(self, store, rev, prev, linknode):
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1044 node = store.node(rev)
205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1045 p1, p2 = store.parentrevs(rev)
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	1046 base = self._deltaparent(store, rev, p1, p2, prev)
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	1047
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	1048 prefix = ''
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1049 if store.iscensored(base) or store.iscensored(rev):
24190 903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1050 try:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1051 delta = store.revision(node, raw=True)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25624 diff changeset	1052 except error.CensoredNodeError as e:
24190 903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1053 delta = e.tombstone
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1054 if base == nullrev:
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1055 prefix = mdiff.trivialdiffheader(len(delta))
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1056 else:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1057 baselen = store.rawsize(base)
24190 903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1058 prefix = mdiff.replacediffheader(baselen, len(delta))
903c7e8c97ad changegroup: emit full-replacement deltas if either revision is censored Mike Edgar <adgar@google.com> parents: 24180 diff changeset	1059 elif base == nullrev:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1060 delta = store.revision(node, raw=True)
14143 da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	1061 prefix = mdiff.trivialdiffheader(len(delta))
da635d3c5620 changegroup: new bundler API Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14141 diff changeset	1062 else:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1063 delta = store.revdiff(base, rev)
205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1064 p1n, p2n = store.parents(node)
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1065
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1066 return revisiondelta(
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1067 node=node,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1068 p1node=p1n,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1069 p2node=p2n,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1070 basenode=store.node(base),
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1071 linknode=linknode,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1072 flags=store.flags(rev),
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1073 deltachunks=(prefix, delta),
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1074 )
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1075
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1076 def _revisiondeltanarrow(self, store, rev, prev, linknode):
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1077 # build up some mapping information that's useful later. See
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1078 # the local() nested function below.
38944 379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	1079 if not self._changelogdone:
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1080 self._clnodetorev[linknode] = rev
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1081 linkrev = rev
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1082 self._clrevtolocalrev[linkrev] = rev
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1083 else:
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1084 linkrev = self._clnodetorev[linknode]
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1085 self._clrevtolocalrev[linkrev] = rev
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1086
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1087 # This is a node to send in full, because the changeset it
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1088 # corresponds to was a full changeset.
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	1089 if linknode in self._full_nodes:
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1090 return self._revisiondeltanormal(store, rev, prev, linknode)
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1091
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1092 # At this point, a node can either be one we should skip or an
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1093 # ellipsis. If it's not an ellipsis, bail immediately.
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1094 if linkrev not in self._precomputedellipsis:
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1095 return
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1096
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1097 linkparents = self._precomputedellipsis[linkrev]
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1098 def local(clrev):
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1099 """Turn a changelog revnum into a local revnum.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1100
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1101 The ellipsis dag is stored as revnums on the changelog,
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1102 but when we're producing ellipsis entries for
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1103 non-changelog revlogs, we need to turn those numbers into
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1104 something local. This does that for us, and during the
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1105 changelog sending phase will also expand the stored
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1106 mappings as needed.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1107 """
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1108 if clrev == nullrev:
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1109 return nullrev
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1110
38944 379d90327861 changegroup: move changelogdone into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38943 diff changeset	1111 if not self._changelogdone:
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1112 # If we're doing the changelog, it's possible that we
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1113 # have a parent that is already on the client, and we
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1114 # need to store some extra mapping information so that
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1115 # our contained ellipsis nodes will be able to resolve
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1116 # their parents.
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1117 if clrev not in self._clrevtolocalrev:
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1118 clnode = store.node(clrev)
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1119 self._clnodetorev[clnode] = clrev
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1120 return clrev
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1121
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1122 # Walk the ellipsis-ized changelog breadth-first looking for a
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1123 # change that has been linked from the current revlog.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1124 #
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1125 # For a flat manifest revlog only a single step should be necessary
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1126 # as all relevant changelog entries are relevant to the flat
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1127 # manifest.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1128 #
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1129 # For a filelog or tree manifest dirlog however not every changelog
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1130 # entry will have been relevant, so we need to skip some changelog
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1131 # nodes even after ellipsis-izing.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1132 walk = [clrev]
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1133 while walk:
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1134 p = walk[0]
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1135 walk = walk[1:]
38945 0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1136 if p in self._clrevtolocalrev:
0548f696795b changegroup: move revision maps to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38944 diff changeset	1137 return self._clrevtolocalrev[p]
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	1138 elif p in self._full_nodes:
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1139 walk.extend([pp for pp in self._repo.changelog.parentrevs(p)
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1140 if pp != nullrev])
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1141 elif p in self._precomputedellipsis:
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1142 walk.extend([pp for pp in self._precomputedellipsis[p]
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1143 if pp != nullrev])
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1144 else:
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1145 # In this case, we've got an ellipsis with parents
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1146 # outside the current bundle (likely an
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1147 # incremental pull). We "know" that we can use the
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1148 # value of this same revlog at whatever revision
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1149 # is pointed to by linknode. "Know" is in scare
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1150 # quotes because I haven't done enough examination
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1151 # of edge cases to convince myself this is really
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1152 # a fact - it works for all the (admittedly
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1153 # thorough) cases in our testsuite, but I would be
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1154 # somewhat unsurprised to find a case in the wild
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1155 # where this breaks down a bit. That said, I don't
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1156 # know if it would hurt anything.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1157 for i in pycompat.xrange(rev, 0, -1):
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1158 if store.linkrev(i) == clrev:
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1159 return i
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1160 # We failed to resolve a parent for this node, so
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1161 # we crash the changegroup construction.
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1162 raise error.Abort(
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1163 'unable to resolve parent while packing %r %r'
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1164 ' for changeset %r' % (store.indexfile, rev, clrev))
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1165
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1166 return nullrev
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1167
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1168 if not linkparents or (
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1169 store.parentrevs(rev) == (nullrev, nullrev)):
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1170 p1, p2 = nullrev, nullrev
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1171 elif len(linkparents) == 1:
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1172 p1, = sorted(local(p) for p in linkparents)
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1173 p2 = nullrev
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1174 else:
66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1175 p1, p2 = sorted(local(p) for p in linkparents)
38931 eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1176
38930 205c98e2f1ba changegroup: rename "revlog" variables Gregory Szorc <gregory.szorc@gmail.com> parents: 38929 diff changeset	1177 n = store.node(rev)
38931 eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1178 p1n, p2n = store.node(p1), store.node(p2)
eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1179 flags = store.flags(rev)
eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1180 flags \|= revlog.REVIDX_ELLIPSIS
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1181
38931 eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1182 # TODO: try and actually send deltas for ellipsis data blocks
eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1183 data = store.revision(n)
eb022ce9e505 changegroup: inline ellipsisdata() Gregory Szorc <gregory.szorc@gmail.com> parents: 38930 diff changeset	1184 diffheader = mdiff.trivialdiffheader(len(data))
38932 23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1185
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1186 return revisiondelta(
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1187 node=n,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1188 p1node=p1n,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1189 p2node=p2n,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1190 basenode=nullid,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1191 linknode=linknode,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1192 flags=flags,
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1193 deltachunks=(diffheader, data),
23d582caae30 changegroup: capture revision delta in a data structure Gregory Szorc <gregory.szorc@gmail.com> parents: 38931 diff changeset	1194 )
38925 66cf046ef60f changegroup: move revchunk() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38924 diff changeset	1195
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1196 def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False,
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1197 shallow=False, ellipsisroots=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1198 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1199 d.node, d.p1node, d.p2node, d.linknode)
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1200
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1201 return cgpacker(repo, filematcher, b'01',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1202 useprevdelta=True,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1203 allowreorder=None,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1204 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1205 manifestsend=b'',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1206 sendtreemanifests=False,
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1207 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1208 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1209 shallow=shallow,
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1210 ellipsisroots=ellipsisroots)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1211
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1212 def _makecg2packer(repo, filematcher, bundlecaps, ellipses=False,
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1213 shallow=False, ellipsisroots=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1214 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1215 d.node, d.p1node, d.p2node, d.basenode, d.linknode)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1216
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	1217 # Since generaldelta is directly supported by cg2, reordering
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	1218 # generally doesn't help, so we disable it by default (treating
6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	1219 # bundle.reorder=auto just like bundle.reorder=False).
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1220 return cgpacker(repo, filematcher, b'02',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1221 useprevdelta=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1222 allowreorder=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1223 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1224 manifestsend=b'',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1225 sendtreemanifests=False,
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1226 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1227 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1228 shallow=shallow,
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1229 ellipsisroots=ellipsisroots)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1230
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1231 def _makecg3packer(repo, filematcher, bundlecaps, ellipses=False,
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1232 shallow=False, ellipsisroots=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1233 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1234 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1235
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1236 return cgpacker(repo, filematcher, b'03',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1237 useprevdelta=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1238 allowreorder=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1239 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1240 manifestsend=closechunk(),
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1241 sendtreemanifests=True,
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1242 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1243 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1244 shallow=shallow,
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1245 ellipsisroots=ellipsisroots)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1246
19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1247 _packermap = {'01': (_makecg1packer, cg1unpacker),
26709 42733e956887 changegroup: reformat packermap and add comment Augie Fackler <augie@google.com> parents: 26708 diff changeset	1248 # cg2 adds support for exchanging generaldelta
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1249 '02': (_makecg2packer, cg2unpacker),
27753 d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	1250 # cg3 adds support for exchanging revlog flags and treemanifests
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1251 '03': (_makecg3packer, cg3unpacker),
26709 42733e956887 changegroup: reformat packermap and add comment Augie Fackler <augie@google.com> parents: 26708 diff changeset	1252 }
23168 a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions Pierre-Yves David <pierre-yves.david@fb.com> parents: 22971 diff changeset	1253
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1254 def allsupportedversions(repo):
27928 c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1255 versions = set(_packermap.keys())
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1256 if not (repo.ui.configbool('experimental', 'changegroup3') or
30633 a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1257 repo.ui.configbool('experimental', 'treemanifest') or
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1258 'treemanifest' in repo.requirements):
30631 438532c99b54 changegroup: simplify 'allsupportedversions' logic Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30594 diff changeset	1259 versions.discard('03')
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1260 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1261
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1262 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1263 def supportedincomingversions(repo):
30633 a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1264 return allsupportedversions(repo)
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1265
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1266 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1267 def supportedoutgoingversions(repo):
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1268 versions = allsupportedversions(repo)
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1269 if 'treemanifest' in repo.requirements:
27928 c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1270 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1271 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1272 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1273 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1274 # support versions 01 and 02.
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1275 versions.discard('01')
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1276 versions.discard('02')
38874 a232e6744ba3 narrow: move requirement constant from changegroup to repository Martin von Zweigbergk <martinvonz@google.com> parents: 38846 diff changeset	1277 if repository.NARROW_REQUIREMENT in repo.requirements:
36495 94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1278 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1279 # support that for stripping and unbundling to work.
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1280 versions.discard('01')
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1281 versions.discard('02')
37135 a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1282 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1283 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1284 # mark LFS entries with REVIDX_EXTSTORED.
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1285 versions.discard('01')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1286 versions.discard('02')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1287
27752 29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config Martin von Zweigbergk <martinvonz@google.com> parents: 27751 diff changeset	1288 return versions
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1289
33676 91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1290 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1291 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1292 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1293 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1294
27929 3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1295 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1296 # Finds the smallest version that it's safe to assume clients of the repo
27931 1289a122cf3f shelve: use cg3 for treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27929 diff changeset	1297 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27929 diff changeset	1298 # support changegroup 02.
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1299 versions = supportedoutgoingversions(repo)
27929 3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1300 if 'generaldelta' in repo.requirements:
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1301 versions.discard('01')
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1302 assert versions
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1303 return min(versions)
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1304
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1305 def getbundler(version, repo, bundlecaps=None, filematcher=None,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1306 ellipses=False, shallow=False, ellipsisroots=None):
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1307 assert version in supportedoutgoingversions(repo)
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1308
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1309 if filematcher is None:
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1310 filematcher = matchmod.alwaysmatcher(repo.root, '')
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1311
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1312 if version == '01' and not filematcher.always():
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1313 raise error.ProgrammingError('version 01 changegroups do not support '
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1314 'sparse file matchers')
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1315
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1316 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1317 raise error.Abort(
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1318 _('ellipsis nodes require at least cg3 on client and server, '
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1319 'but negotiated version %s') % version)
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1320
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1321 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1322 # filter those out.
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1323 filematcher = matchmod.intersectmatchers(repo.narrowmatch(),
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1324 filematcher)
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1325
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1326 fn = _packermap[version][0]
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1327 return fn(repo, filematcher, bundlecaps, ellipses=ellipses,
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1328 shallow=shallow, ellipsisroots=ellipsisroots)
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1329
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	1330 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	1331 return _packermap[version][1](fh, alg, extras=extras)
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1332
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1333 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1334 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1335 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1336 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1337 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1338 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1339 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1340
34111 f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1341 def makechangegroup(repo, outgoing, version, source, fastpath=False,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1342 bundlecaps=None):
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1343 cgstream = makestream(repo, outgoing, version, source,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1344 fastpath=fastpath, bundlecaps=bundlecaps)
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1345 return getunbundler(version, util.chunkbuffer(cgstream), None,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1346 {'clcount': len(outgoing.missing) })
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1347
34118 92f1e2be8ab6 changegroup: rename getsubsetraw to makestream Durham Goode <durham@fb.com> parents: 34116 diff changeset	1348 def makestream(repo, outgoing, version, source, fastpath=False,
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1349 bundlecaps=None, filematcher=None):
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1350 bundler = getbundler(version, repo, bundlecaps=bundlecaps,
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1351 filematcher=filematcher)
34118 92f1e2be8ab6 changegroup: rename getsubsetraw to makestream Durham Goode <durham@fb.com> parents: 34116 diff changeset	1352
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1353 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1354 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1355 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1356 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1357 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1358 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1359 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1360 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1361 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1362 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1363
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1364 repo.hook('preoutgoing', throw=True, source=source)
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1365 _changegroupinfo(repo, csets, source)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	1366 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	1367
28361 277a22cd8741 changegroup: progress for added files is not measured in "chunks" Martin von Zweigbergk <martinvonz@google.com> parents: 28360 diff changeset	1368 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1369 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1370 files = 0
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1371 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1372 total=expectedfiles)
29736 4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29704 diff changeset	1373 for chunkdata in iter(source.filelogheader, {}):
28361 277a22cd8741 changegroup: progress for added files is not measured in "chunks" Martin von Zweigbergk <martinvonz@google.com> parents: 28360 diff changeset	1374 files += 1
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1375 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1376 repo.ui.debug("adding %s revisions\n" % f)
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1377 progress.increment()
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	1378 fl = repo.file(f)
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1379 o = len(fl)
24120 a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	1380 try:
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	1381 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	1382 if not fl.addgroup(deltas, revmap, trp):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1383 raise error.Abort(_("received file revlog group is empty"))
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25624 diff changeset	1384 except error.CensoredBaseError as e:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1385 raise error.Abort(_("received delta base is censored: %s") % e)
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	1386 revisions += len(fl) - o
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1387 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1388 needs = needfiles[f]
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	1389 for new in pycompat.xrange(o, len(fl)):
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1390 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1391 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1392 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1393 else:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1394 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1395 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1396 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1397 del needfiles[f]
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1398 progress.complete()
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1399
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1400 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1401 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1402 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1403 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1404 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1405 except error.LookupError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1406 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1407 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1408 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1409
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1410 return revisions, files
38923 a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1411
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1412 def _packellipsischangegroup(repo, common, match, relevant_nodes,
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1413 ellipsisroots, visitnodes, depth, source, version):
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1414 # We wrap cg1packer.revchunk, using a side channel to pass
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1415 # relevant_nodes into that area. Then if linknode isn't in the
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1416 # set, we know we have an ellipsis node and we should defer
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1417 # sending that node's data. We override close() to detect
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1418 # pending ellipsis nodes and flush them.
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1419 packer = getbundler(version, repo, filematcher=match,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1420 ellipses=True,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1421 shallow=depth is not None,
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1422 ellipsisroots=ellipsisroots)
38923 a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1423 # Give the packer the list of nodes which should not be
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1424 # ellipsis nodes. We store this rather than the set of nodes
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1425 # that should be an ellipsis because for very large histories
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1426 # we expect this to be significantly smaller.
38942 68e490ed640e changegroup: make some packer attributes private Gregory Szorc <gregory.szorc@gmail.com> parents: 38941 diff changeset	1427 packer._full_nodes = relevant_nodes
38923 a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1428
a2f521773761 changegroup: move _packellipsischangegroup() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38922 diff changeset	1429 return packer.generate(common, visitnodes, False, source)

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 38947:1469584ad5fe