mercurial-scm/hg-stable: mercurial/changegroup.py annotate

annotate mercurial/changegroup.py @ 40497:968dd7e02ac5

changegroup: allow to force delta to be against p1 This new developer option is useful to general more "generic" bundle. Without this option, a bundle generated from the repository use deltas similar to the one stored in the specific repository it was generated from. This makes performance testing a bit tricky. Using deltas similar to the final result means all delta stored in the bundle can be applied to the target repository without any further processing (except for the rare case of a full snapshot). The application of such bundles (almost) never exercises the (slower) path of searching for a new valid delta. This result in unrealistic and too favorable timing and profile. Instead, we introduce an option to make sure all revisions are stored as a delta against p1. It might not be the best generation option, but it guarantees that the content will be "generic", not favoring a specific target.

author	Boris Feld <boris.feld@octobus.net>
date	Wed, 10 Oct 2018 00:21:02 +0200
parents	6a917075535a
children	808b762679cd

rev	line source
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9437 diff changeset	6 # GNU General Public License version 2 or any later version.
3877 abaee83ce0a6 Replace demandload with new demandimport Matt Mackall <mpm@selenic.com> parents: 3859 diff changeset	7
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	9
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	10 import os
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	11 import struct
20933 d3775db748a0 localrepo: move the addchangegroup method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20932 diff changeset	12 import weakref
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	13
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	14 from .i18n import _
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	15 from .node import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	16 hex,
38922 ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38874 diff changeset	17 nullid,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	18 nullrev,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	19 short,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	20 )
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	21
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	22 from . import (
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	23 error,
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	24 match as matchmod,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	25 mdiff,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	26 phases,
30945 82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2) Pulkit Goyal <7895pulkit@gmail.com> parents: 30743 diff changeset	27 pycompat,
38874 a232e6744ba3 narrow: move requirement constant from changegroup to repository Martin von Zweigbergk <martinvonz@google.com> parents: 38846 diff changeset	28 repository,
25921 74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	29 util,
74b303a637bc changegroup: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25831 diff changeset	30 )
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	31
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	32 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	33 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	34 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	35
37135 a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	36 LFS_REQUIREMENT = 'lfs'
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	37
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35051 diff changeset	38 readexactly = util.readexactly
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	39
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	40 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	41 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	42 d = readexactly(stream, 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	43 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	44 if l <= 4:
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	45 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	46 raise error.Abort(_("invalid chunk length %d") % l)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	47 return ""
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	48 return readexactly(stream, l - 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	49
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	50 def chunkheader(length):
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	51 """return a changegroup chunk header (string)"""
5368 61462e7d62ed changegroup: avoid large copies Matt Mackall <mpm@selenic.com> parents: 3932 diff changeset	52 return struct.pack(">l", length + 4)
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	53
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	54 def closechunk():
9437 1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush(). Greg Ward <greg-hg@gerg.ca> parents: 9087 diff changeset	55 """return a changegroup chunk header (string) for a zero-length chunk"""
1981 736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	56 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module. Thomas Arendsen Hein <thomas@intevation.de> parents: diff changeset	57
39019 227ebd88ce5e changegroup: pull _fileheader out of cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 39018 diff changeset	58 def _fileheader(path):
227ebd88ce5e changegroup: pull _fileheader out of cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 39018 diff changeset	59 """Obtain a changegroup chunk header for a named path."""
227ebd88ce5e changegroup: pull _fileheader out of cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 39018 diff changeset	60 return chunkheader(len(path)) + path
227ebd88ce5e changegroup: pull _fileheader out of cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 39018 diff changeset	61
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	62 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	63 """Write chunks to a file and return its filename.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	64
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	65 The stream is assumed to be a bundle file.
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	66 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	67 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	68 """
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	69 fh = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	70 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	71 try:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	72 if filename:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	73 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	74 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	75 else:
30212 260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	76 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	77 # small (4k is common on Linux).
260af19891f2 changegroup: increase write buffer size to 128k Gregory Szorc <gregory.szorc@gmail.com> parents: 30211 diff changeset	78 fh = open(filename, "wb", 131072)
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	79 else:
38197 aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path Yuya Nishihara <yuya@tcha.org> parents: 37339 diff changeset	80 fd, filename = pycompat.mkstemp(prefix="hg-bundle-", suffix=".hg")
36843 5bc7ff103081 py3: use r'' instead of sysstr('') to get around code transformer Yuya Nishihara <yuya@tcha.org> parents: 36768 diff changeset	81 fh = os.fdopen(fd, r"wb")
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	82 cleanup = filename
26540 7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	83 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26509 diff changeset	84 fh.write(c)
3659 025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	85 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	86 return filename
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	87 finally:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	88 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	89 fh.close()
025f68f22ae2 move write_bundle to changegroup.py Matt Mackall <mpm@selenic.com> parents: 2470 diff changeset	90 if cleanup is not None:
20976 c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	91 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	92 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	93 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20966 diff changeset	94 os.unlink(cleanup)
3660 8500a13ec44b create a readbundle function Matt Mackall <mpm@selenic.com> parents: 3659 diff changeset	95
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	96 class cg1unpacker(object):
26708 749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	97 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	98
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	99 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	100 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	101 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	102
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	103 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	104 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	105 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	106 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	107
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	108 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	109 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	110
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	111 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	112 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	113 """
22390 e2806b8613ca changegroup: rename bundle-related functions and classes Sune Foldager <cryo@cyanite.org> parents: 22070 diff changeset	114 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	115 deltaheadersize = deltaheader.size
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	116 version = '01'
27920 da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	117 _grouplistcount = 1 # One list of files after the manifests
da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	118
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	119 def __init__(self, fh, alg, extras=None):
30364 a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	120 if alg is None:
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	121 alg = 'UN'
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	122 if alg not in util.compengines.supportedbundletypes:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	123 raise error.Abort(_('unknown stream compression type: %s')
26266 1e042e31bd0c changegroup: move all compressions utilities in util Pierre-Yves David <pierre-yves.david@fb.com> parents: 25921 diff changeset	124 % alg)
26392 127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	125 if alg == 'BZ':
127b59787fd5 changegroup: use a different compression key for BZ in HG10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 26272 diff changeset	126 alg = '_truncatedBZ'
30364 a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	127
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	128 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API Gregory Szorc <gregory.szorc@gmail.com> parents: 30349 diff changeset	129 self._stream = compengine.decompressorreader(fh)
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	130 self._type = alg
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	131 self.extras = extras or {}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	132 self.callback = None
26706 8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	133
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	134 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist Augie Fackler <augie@google.com> parents: 26704 diff changeset	135 # be used by bundlerepo, but it's a little hard to tell.
12044 bcc7139521b7 bundlerepo: remove duplication of bundle decompressors Matt Mackall <mpm@selenic.com> parents: 12043 diff changeset	136 def compressed(self):
30594 182cacaa4c32 cg1packer: fix `compressed` method Stanislau Hlebik <stash@fb.com> parents: 30364 diff changeset	137 return self._type is not None and self._type != 'UN'
12043 bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	138 def read(self, l):
bef5effb3db0 bundle: introduce bundle class Matt Mackall <mpm@selenic.com> parents: 12042 diff changeset	139 return self._stream.read(l)
12330 e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	140 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	141 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable Matt Mackall <mpm@selenic.com> parents: 12329 diff changeset	142 def tell(self):
12332 680fe77ab5b8 bundlerepo: use bundle objects everywhere Matt Mackall <mpm@selenic.com> parents: 12330 diff changeset	143 return self._stream.tell()
12347 6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	144 def close(self):
6277a9469dff bundlerepo: restore close() method Matt Mackall <mpm@selenic.com> parents: 12336 diff changeset	145 return self._stream.close()
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	146
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	147 def _chunklength(self):
13459 acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34 Jim Hague <jim.hague@acm.org> parents: 13458 diff changeset	148 d = readexactly(self._stream, 4)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	149 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	150 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	151 if l:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	152 raise error.Abort(_("invalid chunk length %d") % l)
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	153 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	154 if self.callback:
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	155 self.callback()
13458 9f2c407caf34 changegroup: don't accept odd chunk headers Mads Kiilerich <mads@kiilerich.com> parents: 13457 diff changeset	156 return l - 4
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	157
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	158 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	159 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	160 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	161
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	162 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	163 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	164 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	165
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	166 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	167 """return the header of the filelogs chunk, v10 only has the filename"""
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	168 l = self._chunklength()
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	169 if not l:
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	170 return {}
3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	171 fname = readexactly(self._stream, l)
20675 f8d50add83e1 changegroup: move from dict() construction to {} literals Augie Fackler <raf@durin42.com> parents: 19708 diff changeset	172 return {'filename': fname}
12334 50946802593d bundle: refactor progress callback Matt Mackall <mpm@selenic.com> parents: 12333 diff changeset	173
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	174 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	175 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	176 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	177 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	178 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	179 deltabase = prevnode
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	180 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	181 return node, p1, p2, deltabase, cs, flags
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	182
14144 3c3c53d8343a unbundler: separate delta and header parsing Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14143 diff changeset	183 def deltachunk(self, prevnode):
26707 5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private Augie Fackler <augie@google.com> parents: 26706 diff changeset	184 l = self._chunklength()
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	185 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	186 return {}
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	187 headerdata = readexactly(self._stream, self.deltaheadersize)
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	188 header = self.deltaheader.unpack(headerdata)
14141 bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14060 diff changeset	189 delta = readexactly(self._stream, l - self.deltaheadersize)
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	190 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
34301 05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	191 return (node, p1, p2, cs, deltabase, delta, flags)
12336 9d234f7d8a77 bundle: move chunk parsing into unbundle class Matt Mackall <mpm@selenic.com> parents: 12335 diff changeset	192
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	193 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	194 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	195
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	196 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	197 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	198 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	199 """
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	200 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	201 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	202 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	203 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	204 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	205 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	206 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	207 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	208 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	209 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	210 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	211
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	212 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	213 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	214 noentries = True
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	215 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	216 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	217 if not chunk:
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	218 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	219 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	220 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	221 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	222 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	223 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	224 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	225 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	226 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	227 parts += 1
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	228 break
34106 bbdca7e460c0 changegroup: fix to allow empty manifest parts Durham Goode <durham@fb.com> parents: 33738 diff changeset	229 noentries = False
20999 1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	230 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	231 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	232 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	233 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	234 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	235 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	236 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20978 diff changeset	237
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	238 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	239 self.callback = prog.increment
26712 04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	240 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	241 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	242 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	243 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	244 self.manifestheader()
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	245 deltas = self.deltaiter()
39272 73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally Gregory Szorc <gregory.szorc@gmail.com> parents: 39271 diff changeset	246 repo.manifestlog.getstorage(b'').addgroup(deltas, revmap, trp)
38379 ef692614e601 progress: hide update(None) in a new complete() method Martin von Zweigbergk <martinvonz@google.com> parents: 38352 diff changeset	247 prog.complete()
28360 11287888ce4b changegroup: exclude submanifests from manifest progress Martin von Zweigbergk <martinvonz@google.com> parents: 28281 diff changeset	248 self.callback = None
26712 04176eaf911b changegroup: move manifest unpacking into its own method Augie Fackler <augie@google.com> parents: 26711 diff changeset	249
33308 248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	250 def apply(self, repo, tr, srctype, url, targetphase=phases.draft,
248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	251 expectedtotal=None):
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	252 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	253 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	254 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	255
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	256 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	257 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	258 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	259 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	260 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	261 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	262 repo = repo.unfiltered()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	263 def csmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	264 repo.ui.debug("add changeset %s\n" % short(x))
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	265 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	266
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	267 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	268 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	269
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	270 changesets = files = revisions = 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	271
26880 fa7f8b686633 changegroup: fix the scope of a try finally Pierre-Yves David <pierre-yves.david@fb.com> parents: 26859 diff changeset	272 try:
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	273 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	274 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	275 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	276 # in this function.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	277 srctype = tr.hookargs.setdefault('source', srctype)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	278 url = tr.hookargs.setdefault('url', url)
33697 38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	279 repo.hook('prechangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	280 throw=True, **pycompat.strkwargs(tr.hookargs))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	281
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	282 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	283 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	284 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	285 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	286 oldheads = set(cl.heads())
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	287
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	288 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	289 # pull off the changeset group
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	290 repo.ui.status(_("adding changesets\n"))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	291 clstart = len(cl)
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	292 progress = repo.ui.makeprogress(_('changesets'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	293 total=expectedtotal)
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	294 self.callback = progress.increment
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	295
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	296 efiles = set()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	297 def onchangelog(cl, node):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	298 efiles.update(cl.readfiles(node))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	299
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	300 self.changelogheader()
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	301 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	302 cgnodes = cl.addgroup(deltas, csmap, trp, addrevisioncb=onchangelog)
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	303 efiles = len(efiles)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	304
33308 248d5890c80a changegroup: remove option to allow empty changegroup (API) Martin von Zweigbergk <martinvonz@google.com> parents: 33227 diff changeset	305 if not cgnodes:
39686 9b07ee0a1054 changegroup: improve the devel-warn to specify changelog was empty Pulkit Goyal <pulkit@yandex-team.ru> parents: 39342 diff changeset	306 repo.ui.develwarn('applied empty changelog from changegroup',
34734 3572b2031cec devel-warn: add 'warn-' to 'devel.empty-changegroup' config Boris Feld <boris.feld@octobus.net> parents: 34301 diff changeset	307 config='warn-empty-changegroup')
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	308 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	309 changesets = clend - clstart
38379 ef692614e601 progress: hide update(None) in a new complete() method Martin von Zweigbergk <martinvonz@google.com> parents: 38352 diff changeset	310 progress.complete()
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	311 self.callback = None
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	312
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	313 # pull off the manifest group
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	314 repo.ui.status(_("adding manifests\n"))
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	315 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	316 # changesets.
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	317 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	318 total=changesets)
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	319 self._unpackmanifests(repo, revmap, trp, progress)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	320
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	321 needfiles = {}
33227 86c9aa1d598f configitems: register the 'server.validate' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33185 diff changeset	322 if repo.ui.configbool('server', 'validate'):
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	323 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	324 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	325 # validate incoming csets have their manifests
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	326 for cset in pycompat.xrange(clstart, clend):
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	327 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	328 mfest = ml[mfnode].readdelta()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	329 # store file cgnodes we must see
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	330 for f, n in mfest.iteritems():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	331 needfiles.setdefault(f, set()).add(n)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	332
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	333 # process the files
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	334 repo.ui.status(_("adding file changes\n"))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	335 newrevs, newfiles = _addchangegroupfiles(
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	336 repo, self, revmap, trp, efiles, needfiles)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	337 revisions += newrevs
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	338 files += newfiles
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	339
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	340 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	341 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	342 heads = cl.heads()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	343 deltaheads = len(heads) - len(oldheads)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	344 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	345 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	346 deltaheads -= 1
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	347 htext = ""
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	348 if deltaheads:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	349 htext = _(" (%+d heads)") % deltaheads
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	350
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	351 repo.ui.status(_("added %d changesets"
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	352 " with %d changes to %d files%s\n")
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	353 % (changesets, revisions, files, htext))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	354 repo.invalidatevolatilesets()
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	355
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	356 if changesets > 0:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	357 if 'node' not in tr.hookargs:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	358 tr.hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	359 tr.hookargs['node_last'] = hex(cl.node(clend - 1))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	360 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	361 else:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	362 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	363 hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	364 hookargs['node_last'] = hex(cl.node(clend - 1))
33697 38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	365 repo.hook('pretxnchangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs Augie Fackler <augie@google.com> parents: 33461 diff changeset	366 throw=True, **pycompat.strkwargs(hookargs))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	367
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	368 added = [cl.node(r) for r in pycompat.xrange(clstart, clend)]
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	369 phaseall = None
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	370 if srctype in ('push', 'serve'):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	371 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	372 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	373 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	374 #
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	375 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	376 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	377 if repo.publishing():
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	378 targetphase = phaseall = phases.public
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	379 else:
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	380 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	381
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	382 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	383 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	384 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	385 # ignored.
33456 ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	386 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	387 if added:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	388 phases.registernew(repo, tr, targetphase, added)
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	389 if phaseall is not None:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew' Boris Feld <boris.feld@octobus.net> parents: 33406 diff changeset	390 phases.advanceboundary(repo, tr, phaseall, cgnodes)
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	391
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	392 if changesets > 0:
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	393
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	394 def runhooks():
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	395 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	396 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	397 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	398 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	399 return
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	400
33738 672ad4f3bb84 changegroup: more *kwargs Augie Fackler <augie@google.com>* parents: 33697 diff changeset	401 repo.hook("changegroup", **pycompat.strkwargs(hookargs))
27867 7ced54ebf972 with: use context manager for transaction in changegroup apply Bryan O'Sullivan <bryano@fb.com> parents: 27754 diff changeset	402
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	403 for n in added:
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	404 args = hookargs.copy()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	405 args['node'] = hex(n)
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	406 del args['node_last']
33738 672ad4f3bb84 changegroup: more *kwargs Augie Fackler <augie@google.com>* parents: 33697 diff changeset	407 repo.hook("incoming", **pycompat.strkwargs(args))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	408
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	409 newheads = [h for h in repo.heads()
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	410 if h not in oldheads]
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	411 repo.ui.log("incoming",
36760 2a5024109490 py3: fix int formatting of "incoming changes" log Yuya Nishihara <yuya@tcha.org> parents: 36495 diff changeset	412 "%d incoming changes - new heads: %s\n",
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	413 len(added),
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	414 ', '.join([hex(c[:6]) for c in newheads]))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	415
32949 b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	416 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
b08431e1b062 changegroup: delete "if True" and reflow Martin von Zweigbergk <martinvonz@google.com> parents: 32948 diff changeset	417 lambda tr: repo._afterlock(runhooks))
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	418 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	419 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	420 # never return 0 here:
32888 b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads" Martin von Zweigbergk <martinvonz@google.com> parents: 32887 diff changeset	421 if deltaheads < 0:
33042 3e102a8dd52c bundle2: record changegroup data in 'op.records' (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32949 diff changeset	422 ret = deltaheads - 1
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	423 else:
33042 3e102a8dd52c bundle2: record changegroup data in 'op.records' (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32949 diff changeset	424 ret = deltaheads + 1
33461 bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply' Boris Feld <boris.feld@octobus.net> parents: 33456 diff changeset	425 return ret
26695 1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() Augie Fackler <augie@google.com> parents: 26694 diff changeset	426
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	427 def deltaiter(self):
34160 c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	428 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	429 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	430
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	431 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	432 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	433 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	434 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34301 05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	435 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	436 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk Durham Goode <durham@fb.com> parents: 34298 diff changeset	437 chain = chunkdata[0]
34160 c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup Durham Goode <durham@fb.com> parents: 34118 diff changeset	438
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	439 class cg2unpacker(cg1unpacker):
26708 749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	440 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	441
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	442 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	443 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	444 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers Augie Fackler <augie@google.com> parents: 26707 diff changeset	445 """
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	446 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	447 deltaheadersize = deltaheader.size
23896 becfecaf9087 changegroup.writebundle: HG2Y support Eric Sumner <ericsumner@fb.com> parents: 23895 diff changeset	448 version = '02'
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	449
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	450 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	451 node, p1, p2, deltabase, cs = headertuple
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	452 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	453 return node, p1, p2, deltabase, cs, flags
23181 832b7ef275c8 changegroup: introduce cg2packer/unpacker Sune Foldager <cryo@cyanite.org> parents: 23178 diff changeset	454
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	455 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	456 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	457
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	458 cg3 streams add support for exchanging treemanifests and revlog
27753 d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	459 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	460 separating manifests and files.
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	461 """
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	462 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38935 271854adc3a6 changegroup: make delta header struct formatters actual structs Gregory Szorc <gregory.szorc@gmail.com> parents: 38934 diff changeset	463 deltaheadersize = deltaheader.size
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	464 version = '03'
27920 da5f23362517 changegroup: cg3 has two empty groups after manifests Martin von Zweigbergk <martinvonz@google.com> parents: 27867 diff changeset	465 _grouplistcount = 2 # One list of manifests and one list of files
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	466
27433 12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	467 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	468 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	469 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header Mike Edgar <adgar@google.com> parents: 27432 diff changeset	470
38352 83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	471 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 38197 diff changeset	472 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29736 4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29704 diff changeset	473 for chunkdata in iter(self.filelogheader, {}):
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	474 # If we get here, there are directory manifests in the changegroup
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	475 d = chunkdata["filename"]
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	476 repo.ui.debug("adding %s revisions\n" % d)
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	477 deltas = self.deltaiter()
39272 73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally Gregory Szorc <gregory.szorc@gmail.com> parents: 39271 diff changeset	478 if not repo.manifestlog.getstorage(d).addgroup(deltas, revmap, trp):
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	479 raise error.Abort(_("received dir revlog group is empty"))
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	480
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	481 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	482 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	483 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	484 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	485 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	486 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	487 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	488 if len(d) < n:
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	489 d += readexactly(self._fh, n - len(d))
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	490 return d
13457 e74fe15dc7fd changegroup: verify all stream reads Mads Kiilerich <mads@kiilerich.com> parents: 13456 diff changeset	491 return readexactly(self._fh, n)
12329 7458de933f26 bundle: push chunkbuffer down into decompress Matt Mackall <mpm@selenic.com> parents: 12044 diff changeset	492
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	493 def _revisiondeltatochunks(delta, headerfn):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	494 """Serialize a revisiondelta to changegroup chunks."""
39054 39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	495
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	496 # The captured revision delta may be encoded as a delta against
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	497 # a base revision or as a full revision. The changegroup format
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	498 # requires that everything on the wire be deltas. So for full
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	499 # revisions, we need to invent a header that says to rewrite
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	500 # data.
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	501
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	502 if delta.delta is not None:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	503 prefix, data = b'', delta.delta
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	504 elif delta.basenode == nullid:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	505 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	506 prefix = mdiff.trivialdiffheader(len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	507 else:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	508 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	509 prefix = mdiff.replacediffheader(delta.baserevisionsize,
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	510 len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	511
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	512 meta = headerfn(delta)
39054 39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	513
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	514 yield chunkheader(len(meta) + len(prefix) + len(data))
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	515 yield meta
39054 39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	516 if prefix:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	517 yield prefix
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas Gregory Szorc <gregory.szorc@gmail.com> parents: 39053 diff changeset	518 yield data
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	519
39035 812eec3f89cb changegroup: remove _clnodetorev Gregory Szorc <gregory.szorc@gmail.com> parents: 39034 diff changeset	520 def _sortnodesellipsis(store, nodes, cl, lookup):
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	521 """Sort nodes for changegroup generation."""
39020 037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	522 # Ellipses serving mode.
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	523 #
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	524 # In a perfect world, we'd generate better ellipsis-ified graphs
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	525 # for non-changelog revlogs. In practice, we haven't started doing
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	526 # that yet, so the resulting DAGs for the manifestlog and filelogs
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	527 # are actually full of bogus parentage on all the ellipsis
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	528 # nodes. This has the side effect that, while the contents are
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	529 # correct, the individual DAGs might be completely out of whack in
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	530 # a case like 882681bc3166 and its ancestors (back about 10
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	531 # revisions or so) in the main hg repo.
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	532 #
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	533 # The one invariant we know holds is that the new (potentially
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	534 # bogus) DAG shape will be valid if we order the nodes in the
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	535 # order that they're introduced in dramatis personae by the
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	536 # changelog, so what we do is we sort the non-changelog histories
037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	537 # by the order in which they are used by the changelog.
39035 812eec3f89cb changegroup: remove _clnodetorev Gregory Szorc <gregory.szorc@gmail.com> parents: 39034 diff changeset	538 key = lambda n: cl.rev(lookup(n))
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	539 return sorted(nodes, key=key)
39020 037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	540
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	541 def _resolvenarrowrevisioninfo(cl, store, ischangelog, rev, linkrev,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	542 linknode, clrevtolocalrev, fullclnodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	543 precomputedellipsis):
39042 e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	544 linkparents = precomputedellipsis[linkrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	545 def local(clrev):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	546 """Turn a changelog revnum into a local revnum.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	547
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	548 The ellipsis dag is stored as revnums on the changelog,
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	549 but when we're producing ellipsis entries for
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	550 non-changelog revlogs, we need to turn those numbers into
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	551 something local. This does that for us, and during the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	552 changelog sending phase will also expand the stored
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	553 mappings as needed.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	554 """
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	555 if clrev == nullrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	556 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	557
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	558 if ischangelog:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	559 return clrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	560
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	561 # Walk the ellipsis-ized changelog breadth-first looking for a
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	562 # change that has been linked from the current revlog.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	563 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	564 # For a flat manifest revlog only a single step should be necessary
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	565 # as all relevant changelog entries are relevant to the flat
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	566 # manifest.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	567 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	568 # For a filelog or tree manifest dirlog however not every changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	569 # entry will have been relevant, so we need to skip some changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	570 # nodes even after ellipsis-izing.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	571 walk = [clrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	572 while walk:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	573 p = walk[0]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	574 walk = walk[1:]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	575 if p in clrevtolocalrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	576 return clrevtolocalrev[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	577 elif p in fullclnodes:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	578 walk.extend([pp for pp in cl.parentrevs(p)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	579 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	580 elif p in precomputedellipsis:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	581 walk.extend([pp for pp in precomputedellipsis[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	582 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	583 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	584 # In this case, we've got an ellipsis with parents
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	585 # outside the current bundle (likely an
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	586 # incremental pull). We "know" that we can use the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	587 # value of this same revlog at whatever revision
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	588 # is pointed to by linknode. "Know" is in scare
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	589 # quotes because I haven't done enough examination
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	590 # of edge cases to convince myself this is really
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	591 # a fact - it works for all the (admittedly
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	592 # thorough) cases in our testsuite, but I would be
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	593 # somewhat unsurprised to find a case in the wild
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	594 # where this breaks down a bit. That said, I don't
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	595 # know if it would hurt anything.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	596 for i in pycompat.xrange(rev, 0, -1):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	597 if store.linkrev(i) == clrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	598 return i
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	599 # We failed to resolve a parent for this node, so
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	600 # we crash the changegroup construction.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	601 raise error.Abort(
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	602 'unable to resolve parent while packing %r %r'
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	603 ' for changeset %r' % (store.indexfile, rev, clrev))
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	604
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	605 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	606
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	607 if not linkparents or (
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	608 store.parentrevs(rev) == (nullrev, nullrev)):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	609 p1, p2 = nullrev, nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	610 elif len(linkparents) == 1:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	611 p1, = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	612 p2 = nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	613 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	614 p1, p2 = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	615
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	616 p1node, p2node = store.node(p1), store.node(p2)
39042 e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	617
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	618 return p1node, p2node, linknode
39042 e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39041 diff changeset	619
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	620 def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev,
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	621 topic=None,
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	622 ellipses=False, clrevtolocalrev=None, fullclnodes=None,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	623 precomputedellipsis=None):
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	624 """Calculate deltas for a set of revisions.
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	625
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	626 Is a generator of ``revisiondelta`` instances.
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	627
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	628 If topic is not None, progress detail will be generated using this
0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	629 topic name (e.g. changesets, manifests, etc).
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	630 """
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	631 if not nodes:
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	632 return
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	633
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	634 cl = repo.changelog
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	635
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	636 if ischangelog:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	637 # `hg log` shows changesets in storage order. To preserve order
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	638 # across clones, send out changesets in storage order.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	639 nodesorder = 'storage'
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	640 elif ellipses:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	641 nodes = _sortnodesellipsis(store, nodes, cl, lookup)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	642 nodesorder = 'nodes'
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	643 else:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	644 nodesorder = None
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	645
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	646 # Perform ellipses filtering and revision massaging. We do this before
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	647 # emitrevisions() because a) filtering out revisions creates less work
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	648 # for emitrevisions() b) dropping revisions would break emitrevisions()'s
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	649 # assumptions about delta choices and we would possibly send a delta
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	650 # referencing a missing base revision.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	651 #
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	652 # Also, calling lookup() has side-effects with regards to populating
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	653 # data structures. If we don't call lookup() for each node or if we call
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	654 # lookup() after the first pass through each node, things can break -
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	655 # possibly intermittently depending on the python hash seed! For that
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	656 # reason, we store a mapping of all linknodes during the initial node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	657 # pass rather than use lookup() on the output side.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	658 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	659 filtered = []
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	660 adjustedparents = {}
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	661 linknodes = {}
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	662
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	663 for node in nodes:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	664 rev = store.rev(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	665 linknode = lookup(node)
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	666 linkrev = cl.rev(linknode)
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	667 clrevtolocalrev[linkrev] = rev
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	668
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	669 # If linknode is in fullclnodes, it means the corresponding
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	670 # changeset was a full changeset and is being sent unaltered.
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	671 if linknode in fullclnodes:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	672 linknodes[node] = linknode
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	673
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	674 # If the corresponding changeset wasn't in the set computed
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	675 # as relevant to us, it should be dropped outright.
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	676 elif linkrev not in precomputedellipsis:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	677 continue
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	678
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	679 else:
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	680 # We could probably do this later and avoid the dict
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	681 # holding state. But it likely doesn't matter.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	682 p1node, p2node, linknode = _resolvenarrowrevisioninfo(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	683 cl, store, ischangelog, rev, linkrev, linknode,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	684 clrevtolocalrev, fullclnodes, precomputedellipsis)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	685
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	686 adjustedparents[node] = (p1node, p2node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	687 linknodes[node] = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	688
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	689 filtered.append(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	690
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	691 nodes = filtered
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	692
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	693 # We expect the first pass to be fast, so we only engage the progress
e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	694 # meter for constructing the revision deltas.
e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	695 progress = None
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	696 if topic is not None:
0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	697 progress = repo.ui.makeprogress(topic, unit=_('chunks'),
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	698 total=len(nodes))
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	699
40497 968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	700 configtarget = repo.ui.config('devel', 'bundle.delta')
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	701 if configtarget not in ('', 'p1'):
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	702 msg = _("""config "devel.bundle.delta" as unknown value: %s""")
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	703 repo.ui.warn(msg % configtarget)
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	704
40496 6a917075535a storage: also use `deltamode argument` for ifiledata Boris Feld <boris.feld@octobus.net> parents: 40344 diff changeset	705 deltamode = repository.CG_DELTAMODE_STD
6a917075535a storage: also use `deltamode argument` for ifiledata Boris Feld <boris.feld@octobus.net> parents: 40344 diff changeset	706 if forcedeltaparentprev:
6a917075535a storage: also use `deltamode argument` for ifiledata Boris Feld <boris.feld@octobus.net> parents: 40344 diff changeset	707 deltamode = repository.CG_DELTAMODE_PREV
40497 968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	708 elif configtarget == 'p1':
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40496 diff changeset	709 deltamode = repository.CG_DELTAMODE_P1
40496 6a917075535a storage: also use `deltamode argument` for ifiledata Boris Feld <boris.feld@octobus.net> parents: 40344 diff changeset	710
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	711 revisions = store.emitrevisions(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	712 nodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	713 nodesorder=nodesorder,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	714 revisiondata=True,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	715 assumehaveparentrevisions=not ellipses,
40496 6a917075535a storage: also use `deltamode argument` for ifiledata Boris Feld <boris.feld@octobus.net> parents: 40344 diff changeset	716 deltamode=deltamode)
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	717
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	718 for i, revision in enumerate(revisions):
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	719 if progress:
e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	720 progress.update(i + 1)
e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	721
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	722 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	723 linknode = linknodes[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	724
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	725 if revision.node in adjustedparents:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	726 p1node, p2node = adjustedparents[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	727 revision.p1node = p1node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	728 revision.p2node = p2node
40048 8e398628a3f2 repository: define and use revision flag constants Gregory Szorc <gregory.szorc@gmail.com> parents: 39871 diff changeset	729 revision.flags \|= repository.REVISION_FLAG_ELLIPSIS
39870 31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	730
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	731 else:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	732 linknode = lookup(revision.node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	733
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	734 revision.linknode = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976) Gregory Szorc <gregory.szorc@gmail.com> parents: 39866 diff changeset	735 yield revision
39056 e793e11e1462 changegroup: introduce requests to define delta generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39055 diff changeset	736
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	737 if progress:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	738 progress.complete()
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	739
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	740 class cgpacker(object):
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	741 def __init__(self, repo, oldmatcher, matcher, version,
39055 ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	742 builddeltaheader, manifestsend,
ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	743 forcedeltaparentprev=False,
38964 8a13855c8dbe changegroup: always use the treemanifest-enabled version of _packmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 38949 diff changeset	744 bundlecaps=None, ellipses=False,
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	745 shallow=False, ellipsisroots=None, fullnodes=None):
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	746 """Given a source repo, construct a bundler.
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	747
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	748 oldmatcher is a matcher that matches on files the client already has.
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	749 These will not be included in the changegroup.
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	750
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	751 matcher is a matcher that matches on files to include in the
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	752 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	753
39055 ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	754 forcedeltaparentprev indicates whether delta parents must be against
ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	755 the previous revision in a delta group. This should only be used for
ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	756 compatibility with changegroup version 1.
38940 23ae0c07a3e1 changegroup: control delta parent behavior via constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38939 diff changeset	757
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	758 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	759 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	760
38937 67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	761 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	762
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	763 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	764
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	765 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	766 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	767 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	768 capabilities to customize the changegroup packer.
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	769
cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	770 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	771 need to pack file contents not introduced by the changes being packed.
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	772
39034 5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	773 fullnodes is the set of changelog nodes which should not be ellipsis
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	774 nodes. We store this rather than the set of nodes that should be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	775 ellipsis because for very large histories we expect this to be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	776 significantly smaller.
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	777 """
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	778 assert oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	779 assert matcher
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	780 self._oldmatcher = oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	781 self._matcher = matcher
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	782
38934 d7ac49c2353c changegroup: pass version into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38933 diff changeset	783 self.version = version
39055 ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	784 self._forcedeltaparentprev = forcedeltaparentprev
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	785 self._builddeltaheader = builddeltaheader
38937 67f37e8a5490 changegroup: pass end of manifests marker into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38936 diff changeset	786 self._manifestsend = manifestsend
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	787 self._ellipses = ellipses
38934 d7ac49c2353c changegroup: pass version into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38933 diff changeset	788
32327 df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	789 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	790 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	791 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back Durham Goode <durham@fb.com> parents: 32308 diff changeset	792 self._bundlecaps = bundlecaps
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	793 self._isshallow = shallow
39034 5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	794 self._fullclnodes = fullnodes
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	795
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	796 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	797 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	798
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	799 self._repo = repo
38939 6e999a2d8fe7 changegroup: control reordering via constructor argument Gregory Szorc <gregory.szorc@gmail.com> parents: 38938 diff changeset	800
23748 4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	801 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	802 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	803 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	804 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle Mads Kiilerich <madski@unity3d.com> parents: 23382 diff changeset	805
39688 a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	806 def generate(self, commonrevs, clnodes, fastpathlinkrev, source,
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	807 changelog=True):
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	808 """Yield a sequence of changegroup byte chunks.
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	809 If changelog is False, changelog data won't be added to changegroup
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	810 """
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	811
19202 0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler Sune Foldager <cryo@cyanite.org> parents: 19201 diff changeset	812 repo = self._repo
24978 f52560c64953 changegroup: drop _changelog and _manifest properties Martin von Zweigbergk <martinvonz@google.com> parents: 24977 diff changeset	813 cl = repo.changelog
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	814
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	815 self._verbosenote(_('uncompressed size of bundle content:\n'))
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	816 size = 0
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	817
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	818 clstate, deltas = self._generatechangelog(cl, clnodes)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	819 for delta in deltas:
39688 a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	820 if changelog:
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	821 for chunk in _revisiondeltatochunks(delta,
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	822 self._builddeltaheader):
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	823 size += len(chunk)
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup Pulkit Goyal <pulkit@yandex-team.ru> parents: 39686 diff changeset	824 yield chunk
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	825
39048 fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	826 close = closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	827 size += len(close)
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	828 yield closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	829
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	830 self._verbosenote(_('%8.i (changelog)\n') % size)
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	831
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	832 clrevorder = clstate['clrevorder']
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	833 manifests = clstate['manifests']
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	834 changedfiles = clstate['changedfiles']
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	835
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	836 # We need to make sure that the linkrev in the changegroup refers to
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	837 # the first changeset that introduced the manifest or file revision.
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	838 # The fastpath is usually safer than the slowpath, because the filelogs
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	839 # are walked in revlog order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	840 #
39866 db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	841 # When taking the slowpath when the manifest revlog uses generaldelta,
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	842 # the manifest may be walked in the "wrong" order. Without 'clrevorder',
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	843 # we would get an incorrect linkrev (see fix in cc0ff93d0c0c).
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	844 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	845 # When taking the fastpath, we are only vulnerable to reordering
39866 db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	846 # of the changelog itself. The changelog never uses generaldelta and is
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	847 # never reordered. To handle this case, we simply take the slowpath,
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	848 # which already has the 'clrevorder' logic. This was also fixed in
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	849 # cc0ff93d0c0c.
db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	850
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	851 # Treemanifests don't work correctly with fastpathlinkrev
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	852 # either, because we don't discover which directory nodes to
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	853 # send along with files. This could probably be fixed.
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	854 fastpathlinkrev = fastpathlinkrev and (
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	855 'treemanifest' not in repo.requirements)
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	856
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	857 fnodes = {} # needed file nodes
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	858
39049 2ebdd265fe8c changegroup: move size tracking and end of manifests to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39048 diff changeset	859 size = 0
39050 c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	860 it = self.generatemanifests(
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	861 commonrevs, clrevorder, fastpathlinkrev, manifests, fnodes, source,
39050 c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	862 clstate['clrevtomanifestrev'])
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	863
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	864 for tree, deltas in it:
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	865 if tree:
39050 c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	866 assert self.version == b'03'
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	867 chunk = _fileheader(tree)
39050 c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	868 size += len(chunk)
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	869 yield chunk
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	870
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	871 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	872 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	873 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	874 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	875 yield chunk
39050 c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	876
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	877 close = closechunk()
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	878 size += len(close)
c921ad9cae08 changegroup: move manifest chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39049 diff changeset	879 yield close
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	880
39049 2ebdd265fe8c changegroup: move size tracking and end of manifests to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39048 diff changeset	881 self._verbosenote(_('%8.i (manifests)\n') % size)
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39048 diff changeset	882 yield self._manifestsend
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39048 diff changeset	883
39021 fbbda9ff3deb changegroup: pass mfdicts properly Gregory Szorc <gregory.szorc@gmail.com> parents: 39020 diff changeset	884 mfdicts = None
fbbda9ff3deb changegroup: pass mfdicts properly Gregory Szorc <gregory.szorc@gmail.com> parents: 39020 diff changeset	885 if self._ellipses and self._isshallow:
fbbda9ff3deb changegroup: pass mfdicts properly Gregory Szorc <gregory.szorc@gmail.com> parents: 39020 diff changeset	886 mfdicts = [(self._repo.manifestlog[n].read(), lr)
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	887 for (n, lr) in manifests.iteritems()]
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	888
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	889 manifests.clear()
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	890 clrevs = set(cl.rev(x) for x in clnodes)
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	891
39051 c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	892 it = self.generatefiles(changedfiles, commonrevs,
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	893 source, mfdicts, fastpathlinkrev,
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	894 fnodes, clrevs)
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	895
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	896 for path, deltas in it:
39051 c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	897 h = _fileheader(path)
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	898 size = len(h)
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	899 yield h
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	900
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	901 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	902 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	903 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	904 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	905 yield chunk
39051 c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	906
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	907 close = closechunk()
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	908 size += len(close)
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	909 yield close
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	910
c4a2d19d393a changegroup: move file chunk emission to generate() Gregory Szorc <gregory.szorc@gmail.com> parents: 39050 diff changeset	911 self._verbosenote(_('%8.i %s\n') % (size, path))
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	912
39040 eb8a0139ace3 changegroup: inline _close() Gregory Szorc <gregory.szorc@gmail.com> parents: 39039 diff changeset	913 yield closechunk()
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	914
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	915 if clnodes:
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	916 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	917
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	918 def _generatechangelog(self, cl, nodes):
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	919 """Generate data for changelog chunks.
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	920
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	921 Returns a 2-tuple of a dict containing state and an iterable of
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	922 byte chunks. The state will not be fully populated until the
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	923 chunk stream has been fully consumed.
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	924 """
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	925 clrevorder = {}
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	926 manifests = {}
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	927 mfl = self._repo.manifestlog
28241 a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	928 changedfiles = set()
39036 40374b4a780f changegroup: track changelog to manifest revision map explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 39035 diff changeset	929 clrevtomanifestrev = {}
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	930
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	931 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	932 # manifest nodes.
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	933 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	934 def lookupcl(x):
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	935 c = cl.changelogrevision(x)
23381 cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462) Durham Goode <durham@fb.com> parents: 23226 diff changeset	936 clrevorder[x] = len(clrevorder)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	937
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	938 if self._ellipses:
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	939 # Only update manifests if x is going to be sent. Otherwise we
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	940 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	941 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	942 # have sent.
39034 5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes Gregory Szorc <gregory.szorc@gmail.com> parents: 39033 diff changeset	943 if (x in self._fullclnodes
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	944 or cl.rev(x) in self._precomputedellipsis):
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	945
3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	946 manifestnode = c.manifest
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	947 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	948 # version.
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	949 manifests.setdefault(manifestnode, x)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	950 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	951 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	952 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	953 # parents)
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	954 clrevtomanifestrev.setdefault(
3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	955 cl.rev(x), mfl.rev(manifestnode))
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	956 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	957 # client requested a shallow clone.
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	958 if self._isshallow:
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	959 changedfiles.update(mfl[c.manifest].read().keys())
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	960 else:
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	961 changedfiles.update(c.files)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	962 else:
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	963 # record the first changeset introducing this manifest version
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	964 manifests.setdefault(c.manifest, x)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	965 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	966 # this manifest.
39265 3634ed953a42 changegroup: clean up changelog callback Gregory Szorc <gregory.szorc@gmail.com> parents: 39264 diff changeset	967 changedfiles.update(c.files)
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	968
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	969 return x
19204 e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19202 diff changeset	970
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	971 state = {
f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	972 'clrevorder': clrevorder,
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	973 'manifests': manifests,
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	974 'changedfiles': changedfiles,
39036 40374b4a780f changegroup: track changelog to manifest revision map explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 39035 diff changeset	975 'clrevtomanifestrev': clrevtomanifestrev,
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	976 }
38929 d706c77449f9 changegroup: move generate() modifications from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38928 diff changeset	977
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	978 gen = deltagroup(
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	979 self._repo, cl, nodes, True, lookupcl,
39055 ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	980 self._forcedeltaparentprev,
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	981 ellipses=self._ellipses,
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	982 topic=_('changesets'),
39047 9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	983 clrevtolocalrev={},
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	984 fullclnodes=self._fullclnodes,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 39046 diff changeset	985 precomputedellipsis=self._precomputedellipsis)
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	986
39014 f7228c907ef4 changegroup: factor changelog chunk generation into own function Gregory Szorc <gregory.szorc@gmail.com> parents: 39013 diff changeset	987 return state, gen
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	988
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	989 def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev,
cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	990 manifests, fnodes, source, clrevtolocalrev):
34161 75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	991 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	992
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	993 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	994 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests Durham Goode <durham@fb.com> parents: 34160 diff changeset	995 """
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	996 repo = self._repo
30308 bce79dfcf5e4 manifest: get rid of manifest.readshallowfast Durham Goode <durham@fb.com> parents: 30282 diff changeset	997 mfl = repo.manifestlog
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	998 tmfnodes = {'': manifests}
28227 1c36cc8e7870 changegroup: extract generatemanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	999
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1000 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1001 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1002 # Returns the linkrev node (collected in lookupcl).
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1003 def makelookupmflinknode(tree, nodes):
28231 3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	1004 if fastpathlinkrev:
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1005 assert not tree
39266 cc957b9dc335 changegroup: rename mfs to manifests Gregory Szorc <gregory.szorc@gmail.com> parents: 39265 diff changeset	1006 return manifests.__getitem__
28231 3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	1007
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1008 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1009 """Callback for looking up the linknode for manifests.
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	1010
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1011 Returns the linkrev node for the specified manifest.
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	1012
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1013 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1014
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1015 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1016 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1017 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1018 to send
27219 beb60a898dd0 changegroup: document manifest linkrev callback some more Augie Fackler <augie@google.com> parents: 27218 diff changeset	1019
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1020 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1021 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1022 treemanifests to send.
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1023 """
35051 d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests Kyle Lippincott <spectral@google.com> parents: 34734 diff changeset	1024 clnode = nodes[x]
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1025 mdata = mfl.get(tree, x).readfast(shallow=True)
28241 a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1026 for p, n, fl in mdata.iterentries():
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1027 if fl == 't': # subdirectory manifest
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1028 subtree = tree + p + '/'
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1029 tmfclnodes = tmfnodes.setdefault(subtree, {})
28241 a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1030 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1031 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1032 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests Martin von Zweigbergk <martinvonz@google.com> parents: 28240 diff changeset	1033 else:
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1034 f = tree + p
28240 1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	1035 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	1036 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	1037 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges Martin von Zweigbergk <martinvonz@google.com> parents: 28232 diff changeset	1038 fclnodes[n] = clnode
27239 65c47779bcb5 changegroup: remove one special case from lookupmflinknode Augie Fackler <augie@google.com> parents: 27238 diff changeset	1039 return clnode
28231 3faba927dd93 changegroup: introduce makelookupmflinknode(dir) Martin von Zweigbergk <martinvonz@google.com> parents: 28230 diff changeset	1040 return lookupmflinknode
19206 6308896b1d4a bundle-ng: simplify bundle10.generate Sune Foldager <cryo@cyanite.org> parents: 19204 diff changeset	1041
28232 829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop Martin von Zweigbergk <martinvonz@google.com> parents: 28231 diff changeset	1042 while tmfnodes:
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1043 tree, nodes = tmfnodes.popitem()
39272 73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally Gregory Szorc <gregory.szorc@gmail.com> parents: 39271 diff changeset	1044 store = mfl.getstorage(tree)
39045 39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1045
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1046 if not self._matcher.visitdir(store.tree[:-1] or '.'):
39749 5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1047 # No nodes to send because this directory is out of
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1048 # the client's view of the repository (probably
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1049 # because of narrow clones).
39045 39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1050 prunednodes = []
39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1051 else:
39749 5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1052 # Avoid sending any manifest nodes we can prove the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1053 # client already has by checking linkrevs. See the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1054 # related comment in generatefiles().
39748 e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1055 prunednodes = self._prunemanifests(store, nodes, commonrevs)
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1056 if tree and not prunednodes:
39043 d56a6b78de3b changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39042 diff changeset	1057 continue
d56a6b78de3b changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39042 diff changeset	1058
39261 8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1059 lookupfn = makelookupmflinknode(tree, nodes)
39020 037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	1060
39052 d662959dc881 changegroup: emit revisiondelta instances from deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39051 diff changeset	1061 deltas = deltagroup(
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	1062 self._repo, store, prunednodes, False, lookupfn,
39866 db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	1063 self._forcedeltaparentprev,
39046 8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1064 ellipses=self._ellipses,
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	1065 topic=_('manifests'),
39046 8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1066 clrevtolocalrev=clrevtolocalrev,
8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1067 fullclnodes=self._fullclnodes,
8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1068 precomputedellipsis=self._precomputedellipsis)
8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1069
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1070 if not self._oldmatcher.visitdir(store.tree[:-1] or '.'):
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1071 yield tree, deltas
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1072 else:
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1073 # 'deltas' is a generator and we need to consume it even if
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1074 # we are not going to send it because a side-effect is that
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1075 # it updates tmdnodes (via lookupfn)
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1076 for d in deltas:
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1077 pass
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1078 if not tree:
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1079 yield tree, []
39048 fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	1080
39748 e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1081 def _prunemanifests(self, store, nodes, commonrevs):
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1082 # This is split out as a separate method to allow filtering
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1083 # commonrevs in extension code.
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1084 #
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1085 # TODO(augie): this shouldn't be required, instead we should
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1086 # make filtering of revisions to send delegated to the store
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1087 # layer.
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1088 frev, flr = store.rev, store.linkrev
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1089 return [n for n in nodes if flr(frev(n)) not in commonrevs]
e03c1a63155c changegroup: tease out a temporary prune method for manifests Augie Fackler <augie@google.com> parents: 39747 diff changeset	1090
24897 5c35a6040352 changegroup: document that 'source' parameter exists for extensions Martin von Zweigbergk <martinvonz@google.com> parents: 24896 diff changeset	1091 # The 'source' parameter is useful for extensions
39037 b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1092 def generatefiles(self, changedfiles, commonrevs, source,
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1093 mfdicts, fastpathlinkrev, fnodes, clrevs):
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1094 changedfiles = [f for f in changedfiles
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1095 if self._matcher(f) and not self._oldmatcher(f)]
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1096
39037 b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1097 if not fastpathlinkrev:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1098 def normallinknodes(unused, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1099 return fnodes.get(fname, {})
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1100 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1101 cln = self._repo.changelog.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1102
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1103 def normallinknodes(store, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1104 flinkrev = store.linkrev
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1105 fnode = store.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1106 revs = ((r, flinkrev(r)) for r in store)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1107 return dict((fnode(r), cln(lr))
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1108 for r, lr in revs if lr in clrevs)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1109
39039 a6e1ff40e335 changegroup: pass clrevtolocalrev to each group Gregory Szorc <gregory.szorc@gmail.com> parents: 39038 diff changeset	1110 clrevtolocalrev = {}
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group Gregory Szorc <gregory.szorc@gmail.com> parents: 39038 diff changeset	1111
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1112 if self._isshallow:
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1113 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1114 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1115 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1116 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1117 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1118
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1119 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1120 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1121 try:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1122 fnode = c.filenode(fname)
39039 a6e1ff40e335 changegroup: pass clrevtolocalrev to each group Gregory Szorc <gregory.szorc@gmail.com> parents: 39038 diff changeset	1123 clrevtolocalrev[c.rev()] = flog.rev(fnode)
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1124 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1125 pass
39037 b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1126 links = normallinknodes(flog, fname)
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1127 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1128 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1129 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1130 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1131 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1132 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1133 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1134 return links
39037 b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1135 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles() Gregory Szorc <gregory.szorc@gmail.com> parents: 39036 diff changeset	1136 linknodes = normallinknodes
38928 a06aab274aef changegroup: move generatefiles() from narrow Gregory Szorc <gregory.szorc@gmail.com> parents: 38927 diff changeset	1137
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1138 repo = self._repo
39267 0617a700ef7b changegroup: change topics during generation Gregory Szorc <gregory.szorc@gmail.com> parents: 39266 diff changeset	1139 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	1140 total=len(changedfiles))
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1141 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1142 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1143 if not filerevlog:
37339 5859800edfc5 changegroup: remove "revlog" from error message Gregory Szorc <gregory.szorc@gmail.com> parents: 37135 diff changeset	1144 raise error.Abort(_("empty or missing file data for %s") %
5859800edfc5 changegroup: remove "revlog" from error message Gregory Szorc <gregory.szorc@gmail.com> parents: 37135 diff changeset	1145 fname)
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1146
39039 a6e1ff40e335 changegroup: pass clrevtolocalrev to each group Gregory Szorc <gregory.szorc@gmail.com> parents: 39038 diff changeset	1147 clrevtolocalrev.clear()
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group Gregory Szorc <gregory.szorc@gmail.com> parents: 39038 diff changeset	1148
19334 95a49112e7ab bundle: move file chunk generation to it's own function Durham Goode <durham@fb.com> parents: 19325 diff changeset	1149 linkrevnodes = linknodes(filerevlog, fname)
19207 a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1150 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1151 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1152 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1153 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 19206 diff changeset	1154
39045 39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1155 frev, flr = filerevlog.rev, filerevlog.linkrev
39749 5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1156 # Skip sending any filenode we know the client already
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1157 # has. This avoids over-sending files relatively
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1158 # inexpensively, so it's not a problem if we under-filter
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years Augie Fackler <augie@google.com> parents: 39748 diff changeset	1159 # here.
39045 39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1160 filenodes = [n for n in linkrevnodes
39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1161 if flr(frev(n)) not in commonrevs]
39f5c7afdc25 changegroup: inline _prune() into call sites Gregory Szorc <gregory.szorc@gmail.com> parents: 39044 diff changeset	1162
39058 0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1163 if not filenodes:
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1164 continue
39020 037debbf869c changegroup: pass sorted revisions into group() (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39019 diff changeset	1165
39058 0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1166 progress.update(i + 1, item=fname)
39046 8c84f1ef949e changegroup: pass all state into group() Gregory Szorc <gregory.szorc@gmail.com> parents: 39045 diff changeset	1167
39058 0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1168 deltas = deltagroup(
39257 2646b8d66b7b changegroup: move node sorting into deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39214 diff changeset	1169 self._repo, filerevlog, filenodes, False, lookupfilelog,
39866 db5501d93bcf changegroup: remove reordering control (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39749 diff changeset	1170 self._forcedeltaparentprev,
39058 0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1171 ellipses=self._ellipses,
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1172 clrevtolocalrev=clrevtolocalrev,
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1173 fullclnodes=self._fullclnodes,
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1174 precomputedellipsis=self._precomputedellipsis)
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1175
0b5f534df82a changegroup: invert conditional and dedent Gregory Szorc <gregory.szorc@gmail.com> parents: 39057 diff changeset	1176 yield fname, deltas
39048 fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup() Gregory Szorc <gregory.szorc@gmail.com> parents: 39047 diff changeset	1177
38416 1c5c4a5dd86d changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38388 diff changeset	1178 progress.complete()
19200 4cfdec944edf bundle-ng: move group into the bundler Sune Foldager <cryo@cyanite.org> parents: 19199 diff changeset	1179
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1180 def _makecg1packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1181 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1182 fullnodes=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1183 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1184 d.node, d.p1node, d.p2node, d.linknode)
27432 77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests Augie Fackler <augie@google.com> parents: 27241 diff changeset	1185
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1186 return cgpacker(repo, oldmatcher, matcher, b'01',
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1187 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1188 manifestsend=b'',
39055 ef3d3a2f9aa5 changegroup: refactor delta parent code Gregory Szorc <gregory.szorc@gmail.com> parents: 39054 diff changeset	1189 forcedeltaparentprev=True,
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1190 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1191 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1192 shallow=shallow,
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1193 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1194 fullnodes=fullnodes)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1195
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1196 def _makecg2packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1197 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1198 fullnodes=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1199 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1200 d.node, d.p1node, d.p2node, d.basenode, d.linknode)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1201
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1202 return cgpacker(repo, oldmatcher, matcher, b'02',
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1203 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1204 manifestsend=b'',
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1205 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1206 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1207 shallow=shallow,
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1208 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1209 fullnodes=fullnodes)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1210
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1211 def _makecg3packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1212 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1213 fullnodes=None):
38936 bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1214 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1215 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38935 diff changeset	1216
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1217 return cgpacker(repo, oldmatcher, matcher, b'03',
38941 4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1218 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38940 diff changeset	1219 manifestsend=closechunk(),
38943 cdb9bc216771 changegroup: declare shallow flag in constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38942 diff changeset	1220 bundlecaps=bundlecaps,
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1221 ellipses=ellipses,
38946 ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor Gregory Szorc <gregory.szorc@gmail.com> parents: 38945 diff changeset	1222 shallow=shallow,
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1223 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1224 fullnodes=fullnodes)
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1225
19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1226 _packermap = {'01': (_makecg1packer, cg1unpacker),
26709 42733e956887 changegroup: reformat packermap and add comment Augie Fackler <augie@google.com> parents: 26708 diff changeset	1227 # cg2 adds support for exchanging generaldelta
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1228 '02': (_makecg2packer, cg2unpacker),
27753 d4071cc73f46 changegroup3: add empty chunk separating directories and files Martin von Zweigbergk <martinvonz@google.com> parents: 27752 diff changeset	1229 # cg3 adds support for exchanging revlog flags and treemanifests
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1230 '03': (_makecg3packer, cg3unpacker),
26709 42733e956887 changegroup: reformat packermap and add comment Augie Fackler <augie@google.com> parents: 26708 diff changeset	1231 }
23168 a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions Pierre-Yves David <pierre-yves.david@fb.com> parents: 22971 diff changeset	1232
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1233 def allsupportedversions(repo):
27928 c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1234 versions = set(_packermap.keys())
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1235 if not (repo.ui.configbool('experimental', 'changegroup3') or
30633 a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1236 repo.ui.configbool('experimental', 'treemanifest') or
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1237 'treemanifest' in repo.requirements):
30631 438532c99b54 changegroup: simplify 'allsupportedversions' logic Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30594 diff changeset	1238 versions.discard('03')
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1239 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1240
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1241 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1242 def supportedincomingversions(repo):
30633 a001cd7296a5 changegroup: simplify logic around enabling changegroup 03 Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30632 diff changeset	1243 return allsupportedversions(repo)
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1244
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1245 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1246 def supportedoutgoingversions(repo):
30632 7ace5304fec5 changegroup: pass 'repo' to allsupportedversions Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30631 diff changeset	1247 versions = allsupportedversions(repo)
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1248 if 'treemanifest' in repo.requirements:
27928 c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1249 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1250 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1251 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1252 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1253 # support versions 01 and 02.
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1254 versions.discard('01')
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27920 diff changeset	1255 versions.discard('02')
38874 a232e6744ba3 narrow: move requirement constant from changegroup to repository Martin von Zweigbergk <martinvonz@google.com> parents: 38846 diff changeset	1256 if repository.NARROW_REQUIREMENT in repo.requirements:
36495 94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1257 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1258 # support that for stripping and unbundling to work.
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1259 versions.discard('01')
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core Martin von Zweigbergk <martinvonz@google.com> parents: 36494 diff changeset	1260 versions.discard('02')
37135 a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1261 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1262 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1263 # mark LFS entries with REVIDX_EXTSTORED.
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1264 versions.discard('01')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1265 versions.discard('02')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py Matt Harbison <matt_harbison@yahoo.com> parents: 37087 diff changeset	1266
27752 29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config Martin von Zweigbergk <martinvonz@google.com> parents: 27751 diff changeset	1267 return versions
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1268
33676 91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1269 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1270 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1271 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1272 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678) Martin von Zweigbergk <martinvonz@google.com> parents: 33461 diff changeset	1273
27929 3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1274 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1275 # Finds the smallest version that it's safe to assume clients of the repo
27931 1289a122cf3f shelve: use cg3 for treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27929 diff changeset	1276 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests Martin von Zweigbergk <martinvonz@google.com> parents: 27929 diff changeset	1277 # support changegroup 02.
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1278 versions = supportedoutgoingversions(repo)
27929 3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1279 if 'generaldelta' in repo.requirements:
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1280 versions.discard('01')
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1281 assert versions
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1282 return min(versions)
3b2ac2115464 changegroup: introduce safeversion() Martin von Zweigbergk <martinvonz@google.com> parents: 27928 diff changeset	1283
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1284 def getbundler(version, repo, bundlecaps=None, oldmatcher=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1285 matcher=None, ellipses=False, shallow=False,
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1286 ellipsisroots=None, fullnodes=None):
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27946 diff changeset	1287 assert version in supportedoutgoingversions(repo)
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1288
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1289 if matcher is None:
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1290 matcher = matchmod.alwaysmatcher(repo.root, '')
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1291 if oldmatcher is None:
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1292 oldmatcher = matchmod.nevermatcher(repo.root, '')
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1293
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1294 if version == '01' and not matcher.always():
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1295 raise error.ProgrammingError('version 01 changegroups do not support '
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1296 'sparse file matchers')
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1297
38947 1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1298 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1299 raise error.Abort(
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1300 _('ellipsis nodes require at least cg3 on client and server, '
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1301 'but negotiated version %s') % version)
1469584ad5fe changegroup: specify ellipses mode explicitly Gregory Szorc <gregory.szorc@gmail.com> parents: 38946 diff changeset	1302
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1303 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1304 # filter those out.
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1305 matcher = repo.narrowmatch(matcher)
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1306
38933 19344024a8e1 changegroup: define functions for creating changegroup packers Gregory Szorc <gregory.szorc@gmail.com> parents: 38932 diff changeset	1307 fn = _packermap[version][0]
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1308 return fn(repo, oldmatcher, matcher, bundlecaps, ellipses=ellipses,
38948 1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1309 shallow=shallow, ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker Gregory Szorc <gregory.szorc@gmail.com> parents: 38947 diff changeset	1310 fullnodes=fullnodes)
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1311
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	1312 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29371 diff changeset	1313 return _packermap[version][1](fh, alg, extras=extras)
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27739 diff changeset	1314
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1315 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1316 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1317 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1318 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1319 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1320 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1321 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1322
34111 f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1323 def makechangegroup(repo, outgoing, version, source, fastpath=False,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1324 bundlecaps=None):
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1325 cgstream = makestream(repo, outgoing, version, source,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1326 fastpath=fastpath, bundlecaps=bundlecaps)
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1327 return getunbundler(version, util.chunkbuffer(cgstream), None,
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1328 {'clcount': len(outgoing.missing) })
f85dfde1731a changegroup: replace getsubset with makechangegroup Durham Goode <durham@fb.com> parents: 34106 diff changeset	1329
34118 92f1e2be8ab6 changegroup: rename getsubsetraw to makestream Durham Goode <durham@fb.com> parents: 34116 diff changeset	1330 def makestream(repo, outgoing, version, source, fastpath=False,
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1331 bundlecaps=None, matcher=None):
38834 1d01cf0416a5 changegroup: move file matcher from narrow extension Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	1332 bundler = getbundler(version, repo, bundlecaps=bundlecaps,
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40083 diff changeset	1333 matcher=matcher)
34118 92f1e2be8ab6 changegroup: rename getsubsetraw to makestream Durham Goode <durham@fb.com> parents: 34116 diff changeset	1334
20925 5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1335 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1336 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1337 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1338 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1339 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1340 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1341 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1342 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1343 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1344 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1345
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20675 diff changeset	1346 repo.hook('preoutgoing', throw=True, source=source)
20926 7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20925 diff changeset	1347 _changegroupinfo(repo, csets, source)
23177 706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	1348 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands Sune Foldager <cryo@cyanite.org> parents: 23168 diff changeset	1349
28361 277a22cd8741 changegroup: progress for added files is not measured in "chunks" Martin von Zweigbergk <martinvonz@google.com> parents: 28360 diff changeset	1350 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1351 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1352 files = 0
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1353 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1354 total=expectedfiles)
29736 4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29704 diff changeset	1355 for chunkdata in iter(source.filelogheader, {}):
28361 277a22cd8741 changegroup: progress for added files is not measured in "chunks" Martin von Zweigbergk <martinvonz@google.com> parents: 28360 diff changeset	1356 files += 1
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1357 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1358 repo.ui.debug("adding %s revisions\n" % f)
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1359 progress.increment()
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	1360 fl = repo.file(f)
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1361 o = len(fl)
24120 a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes Mike Edgar <adgar@google.com> parents: 23897 diff changeset	1362 try:
34298 1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	1363 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup Durham Goode <durham@fb.com> parents: 34264 diff changeset	1364 if not fl.addgroup(deltas, revmap, trp):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1365 raise error.Abort(_("received file revlog group is empty"))
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25624 diff changeset	1366 except error.CensoredBaseError as e:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1367 raise error.Abort(_("received delta base is censored: %s") % e)
27754 a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests() Martin von Zweigbergk <martinvonz@google.com> parents: 27753 diff changeset	1368 revisions += len(fl) - o
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1369 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1370 needs = needfiles[f]
38823 e7aa113b14f7 global: use pycompat.xrange() Gregory Szorc <gregory.szorc@gmail.com> parents: 38557 diff changeset	1371 for new in pycompat.xrange(o, len(fl)):
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1372 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1373 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1374 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1375 else:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1376 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1377 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1378 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1379 del needfiles[f]
38388 daa08d45740f changegroup: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38379 diff changeset	1380 progress.complete()
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1381
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1382 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1383 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1384 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1385 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1386 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1387 except error.LookupError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26540 diff changeset	1388 raise error.Abort(
20932 0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1389 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1390 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1391
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module Pierre-Yves David <pierre-yves.david@fb.com> parents: 20931 diff changeset	1392 return revisions, files

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 40497:968dd7e02ac5