Mercurial > public > mercurial-scm > hg

annotate mercurial/changegroup.py @ 39230:b518d495a560

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
repository: formalize interfaces for revision deltas and requests Now that we've sufficiently abstracted how revision deltas are produced in changegroup code, we can now start the process of formalizing that as part of the interfaces defined in the repository module. This commit essentially converts the revisiondelta and revisiondeltarequest classes into well-defined interfaces. This is not strictly necessary. But I want all types formalized by the storage interface to have interfaces. This makes it much easier to test for interface conformance and for implementing new storage backends. Because the interface is documented, comments and docstrings from changegroup.py have been dropped. Differential Revision: https://phab.mercurial-scm.org/D4225
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 09 Aug 2018 15:40:14 -0700
parents 2646b8d66b7b
children b41d023a412a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
4 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9437
diff changeset
6 # GNU General Public License version 2 or any later version.
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3859
diff changeset
7
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
9
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
10 import os
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
11 import struct
20933
d3775db748a0 localrepo: move the addchangegroup method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20932
diff changeset
12 import weakref
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
13
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
14 from .i18n import _
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
15 from .node import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
16 hex,
38883
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38835
diff changeset
17 nullid,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
18 nullrev,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
19 short,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
20 )
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
21
38893
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
22 from .thirdparty import (
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
23 attr,
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
24 )
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
25
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
26 from . import (
39181
0a934ee94f09 dagop: port revlogdag.linearize() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
27 dagop,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
28 error,
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
29 match as matchmod,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
30 mdiff,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
31 phases,
30925
82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30743
diff changeset
32 pycompat,
38835
a232e6744ba3 narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents: 38806
diff changeset
33 repository,
38883
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38835
diff changeset
34 revlog,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
35 util,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
36 )
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
37
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36835
diff changeset
38 from .utils import (
39230
b518d495a560 repository: formalize interfaces for revision deltas and requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39229
diff changeset
39 interfaceutil,
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36835
diff changeset
40 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36835
diff changeset
41 )
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36835
diff changeset
42
38896
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
43 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
44 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
45 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
46
37132
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
47 LFS_REQUIREMENT = 'lfs'
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
48
35754
fb0be099063f util: move 'readexactly' in the util module
Boris Feld <boris.feld@octobus.net>
parents: 35012
diff changeset
49 readexactly = util.readexactly
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
50
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
51 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
52 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
53 d = readexactly(stream, 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
54 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
55 if l <= 4:
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
56 if l:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
57 raise error.Abort(_("invalid chunk length %d") % l)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
58 return ""
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
59 return readexactly(stream, l - 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
60
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
61 def chunkheader(length):
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
62 """return a changegroup chunk header (string)"""
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
63 return struct.pack(">l", length + 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
64
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
65 def closechunk():
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
66 """return a changegroup chunk header (string) for a zero-length chunk"""
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
67 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
68
38981
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38980
diff changeset
69 def _fileheader(path):
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38980
diff changeset
70 """Obtain a changegroup chunk header for a named path."""
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38980
diff changeset
71 return chunkheader(len(path)) + path
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38980
diff changeset
72
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
73 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
74 """Write chunks to a file and return its filename.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
75
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
76 The stream is assumed to be a bundle file.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
77 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
78 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
79 """
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
80 fh = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
81 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
82 try:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
83 if filename:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
84 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
85 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
86 else:
30212
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
87 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
88 # small (4k is common on Linux).
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
89 fh = open(filename, "wb", 131072)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
90 else:
38164
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37339
diff changeset
91 fd, filename = pycompat.mkstemp(prefix="hg-bundle-", suffix=".hg")
36835
5bc7ff103081 py3: use r'' instead of sysstr('') to get around code transformer
Yuya Nishihara <yuya@tcha.org>
parents: 36760
diff changeset
92 fh = os.fdopen(fd, r"wb")
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
93 cleanup = filename
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
94 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
95 fh.write(c)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
96 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
97 return filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
98 finally:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
99 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
100 fh.close()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
101 if cleanup is not None:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
102 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
103 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
104 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
105 os.unlink(cleanup)
3660
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
106
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
107 class cg1unpacker(object):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
108 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
109
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
110 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
111 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
112 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
113
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
114 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
115 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
116 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
117 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
118
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
119 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
120 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
121
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
122 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
123 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
124 """
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
125 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38896
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
126 deltaheadersize = deltaheader.size
23896
becfecaf9087 changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents: 23895
diff changeset
127 version = '01'
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
128 _grouplistcount = 1 # One list of files after the manifests
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
129
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
130 def __init__(self, fh, alg, extras=None):
30354
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
131 if alg is None:
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
132 alg = 'UN'
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
133 if alg not in util.compengines.supportedbundletypes:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
134 raise error.Abort(_('unknown stream compression type: %s')
26266
1e042e31bd0c changegroup: move all compressions utilities in util
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25921
diff changeset
135 % alg)
26392
127b59787fd5 changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26272
diff changeset
136 if alg == 'BZ':
127b59787fd5 changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26272
diff changeset
137 alg = '_truncatedBZ'
30354
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
138
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
139 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30339
diff changeset
140 self._stream = compengine.decompressorreader(fh)
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
141 self._type = alg
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
142 self.extras = extras or {}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
143 self.callback = None
26706
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
144
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
145 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
146 # be used by bundlerepo, but it's a little hard to tell.
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
147 def compressed(self):
30589
182cacaa4c32 cg1packer: fix `compressed` method
Stanislau Hlebik <stash@fb.com>
parents: 30354
diff changeset
148 return self._type is not None and self._type != 'UN'
12043
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
149 def read(self, l):
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
150 return self._stream.read(l)
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
151 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
152 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
153 def tell(self):
12332
680fe77ab5b8 bundlerepo: use bundle objects everywhere
Matt Mackall <mpm@selenic.com>
parents: 12330
diff changeset
154 return self._stream.tell()
12347
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
155 def close(self):
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
156 return self._stream.close()
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
157
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
158 def _chunklength(self):
13459
acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34
Jim Hague <jim.hague@acm.org>
parents: 13458
diff changeset
159 d = readexactly(self._stream, 4)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
160 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
161 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
162 if l:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
163 raise error.Abort(_("invalid chunk length %d") % l)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
164 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
165 if self.callback:
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
166 self.callback()
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
167 return l - 4
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
168
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
169 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
170 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
171 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
172
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
173 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
174 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
175 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
176
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
177 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
178 """return the header of the filelogs chunk, v10 only has the filename"""
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
179 l = self._chunklength()
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
180 if not l:
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
181 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
182 fname = readexactly(self._stream, l)
20675
f8d50add83e1 changegroup: move from dict() construction to {} literals
Augie Fackler <raf@durin42.com>
parents: 19708
diff changeset
183 return {'filename': fname}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
184
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
185 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
186 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
187 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
188 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
189 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
190 deltabase = prevnode
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
191 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
192 return node, p1, p2, deltabase, cs, flags
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
193
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
194 def deltachunk(self, prevnode):
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
195 l = self._chunklength()
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
196 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
197 return {}
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
198 headerdata = readexactly(self._stream, self.deltaheadersize)
38896
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
199 header = self.deltaheader.unpack(headerdata)
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
200 delta = readexactly(self._stream, l - self.deltaheadersize)
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
201 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
34294
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34291
diff changeset
202 return (node, p1, p2, cs, deltabase, delta, flags)
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
203
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
204 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
205 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
206
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
207 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
208 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
209 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
210 """
34091
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
211 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
212 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
213 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
214 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
215 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
216 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
217 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
218 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
219 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
220 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
221 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
222
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
223 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
224 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
225 noentries = True
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
226 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
227 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
228 if not chunk:
34091
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
229 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
230 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
231 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
232 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
233 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
234 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
235 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
236 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
237 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
238 parts += 1
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
239 break
34091
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33712
diff changeset
240 noentries = False
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
241 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
242 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
243 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
244 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
245 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
246 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
247 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
248
38346
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
249 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
250 self.callback = prog.increment
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
251 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
252 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
253 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
254 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
255 self.manifestheader()
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
256 deltas = self.deltaiter()
38556
0db41eb0a3ac manifest: define and implement addgroup() on manifestlog
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38410
diff changeset
257 repo.manifestlog.addgroup(deltas, revmap, trp)
38373
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38346
diff changeset
258 prog.complete()
28360
11287888ce4b changegroup: exclude submanifests from manifest progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 28281
diff changeset
259 self.callback = None
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
260
33308
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
261 def apply(self, repo, tr, srctype, url, targetphase=phases.draft,
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
262 expectedtotal=None):
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
263 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
264 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
265 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
266
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
267 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
268 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
269 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
270 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
271 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
272 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
273 repo = repo.unfiltered()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
274 def csmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
275 repo.ui.debug("add changeset %s\n" % short(x))
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
276 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
277
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
278 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
279 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
280
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
281 changesets = files = revisions = 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
282
26880
fa7f8b686633 changegroup: fix the scope of a try finally
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26859
diff changeset
283 try:
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
284 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
285 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
286 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
287 # in this function.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
288 srctype = tr.hookargs.setdefault('source', srctype)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
289 url = tr.hookargs.setdefault('url', url)
33671
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
290 repo.hook('prechangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
291 throw=True, **pycompat.strkwargs(tr.hookargs))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
292
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
293 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
294 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
295 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
296 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
297 oldheads = set(cl.heads())
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
298
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
299 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
300 # pull off the changeset group
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
301 repo.ui.status(_("adding changesets\n"))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
302 clstart = len(cl)
38346
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
303 progress = repo.ui.makeprogress(_('changesets'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
304 total=expectedtotal)
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
305 self.callback = progress.increment
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
306
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
307 efiles = set()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
308 def onchangelog(cl, node):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
309 efiles.update(cl.readfiles(node))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
310
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
311 self.changelogheader()
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
312 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
313 cgnodes = cl.addgroup(deltas, csmap, trp, addrevisioncb=onchangelog)
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
314 efiles = len(efiles)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
315
33308
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
316 if not cgnodes:
33309
69c4493a54f9 changegroup: don't fail on empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33308
diff changeset
317 repo.ui.develwarn('applied empty changegroup',
34734
3572b2031cec devel-warn: add 'warn-' to 'devel.empty-changegroup' config
Boris Feld <boris.feld@octobus.net>
parents: 34294
diff changeset
318 config='warn-empty-changegroup')
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
319 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
320 changesets = clend - clstart
38373
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38346
diff changeset
321 progress.complete()
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
322 self.callback = None
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
323
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
324 # pull off the manifest group
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
325 repo.ui.status(_("adding manifests\n"))
38346
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
326 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
327 # changesets.
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
328 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
329 total=changesets)
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
330 self._unpackmanifests(repo, revmap, trp, progress)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
331
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
332 needfiles = {}
33227
86c9aa1d598f configitems: register the 'server.validate' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33183
diff changeset
333 if repo.ui.configbool('server', 'validate'):
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
334 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
335 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
336 # validate incoming csets have their manifests
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38556
diff changeset
337 for cset in pycompat.xrange(clstart, clend):
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
338 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
339 mfest = ml[mfnode].readdelta()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
340 # store file cgnodes we must see
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
341 for f, n in mfest.iteritems():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
342 needfiles.setdefault(f, set()).add(n)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
343
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
344 # process the files
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
345 repo.ui.status(_("adding file changes\n"))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
346 newrevs, newfiles = _addchangegroupfiles(
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
347 repo, self, revmap, trp, efiles, needfiles)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
348 revisions += newrevs
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
349 files += newfiles
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
350
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
351 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
352 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
353 heads = cl.heads()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
354 deltaheads = len(heads) - len(oldheads)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
355 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
356 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
357 deltaheads -= 1
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
358 htext = ""
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
359 if deltaheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
360 htext = _(" (%+d heads)") % deltaheads
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
361
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
362 repo.ui.status(_("added %d changesets"
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
363 " with %d changes to %d files%s\n")
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
364 % (changesets, revisions, files, htext))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
365 repo.invalidatevolatilesets()
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
366
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
367 if changesets > 0:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
368 if 'node' not in tr.hookargs:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
369 tr.hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
370 tr.hookargs['node_last'] = hex(cl.node(clend - 1))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
371 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
372 else:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
373 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
374 hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
375 hookargs['node_last'] = hex(cl.node(clend - 1))
33671
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
376 repo.hook('pretxnchangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
377 throw=True, **pycompat.strkwargs(hookargs))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
378
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38556
diff changeset
379 added = [cl.node(r) for r in pycompat.xrange(clstart, clend)]
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
380 phaseall = None
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
381 if srctype in ('push', 'serve'):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
382 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
383 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
384 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
385 #
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
386 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
387 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
388 if repo.publishing():
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
389 targetphase = phaseall = phases.public
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
390 else:
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
391 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
392
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
393 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
394 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
395 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
396 # ignored.
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
397 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
398 if added:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
399 phases.registernew(repo, tr, targetphase, added)
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
400 if phaseall is not None:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
401 phases.advanceboundary(repo, tr, phaseall, cgnodes)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
402
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
403 if changesets > 0:
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
404
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
405 def runhooks():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
406 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
407 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
408 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
409 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
410 return
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
411
33712
672ad4f3bb84 changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents: 33671
diff changeset
412 repo.hook("changegroup", **pycompat.strkwargs(hookargs))
27867
7ced54ebf972 with: use context manager for transaction in changegroup apply
Bryan O'Sullivan <bryano@fb.com>
parents: 27754
diff changeset
413
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
414 for n in added:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
415 args = hookargs.copy()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
416 args['node'] = hex(n)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
417 del args['node_last']
33712
672ad4f3bb84 changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents: 33671
diff changeset
418 repo.hook("incoming", **pycompat.strkwargs(args))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
419
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
420 newheads = [h for h in repo.heads()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
421 if h not in oldheads]
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
422 repo.ui.log("incoming",
36737
2a5024109490 py3: fix int formatting of "incoming changes" log
Yuya Nishihara <yuya@tcha.org>
parents: 36465
diff changeset
423 "%d incoming changes - new heads: %s\n",
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
424 len(added),
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
425 ', '.join([hex(c[:6]) for c in newheads]))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
426
32931
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
427 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32930
diff changeset
428 lambda tr: repo._afterlock(runhooks))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
429 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
430 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
431 # never return 0 here:
32870
b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32869
diff changeset
432 if deltaheads < 0:
33030
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32931
diff changeset
433 ret = deltaheads - 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
434 else:
33030
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32931
diff changeset
435 ret = deltaheads + 1
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33456
diff changeset
436 return ret
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
437
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
438 def deltaiter(self):
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
439 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
440 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
441
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
442 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
443 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
444 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
445 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34294
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34291
diff changeset
446 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34291
diff changeset
447 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34291
diff changeset
448 chain = chunkdata[0]
34148
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34103
diff changeset
449
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
450 class cg2unpacker(cg1unpacker):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
451 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
452
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
453 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
454 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
455 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
456 """
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
457 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38896
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
458 deltaheadersize = deltaheader.size
23896
becfecaf9087 changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents: 23895
diff changeset
459 version = '02'
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
460
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
461 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
462 node, p1, p2, deltabase, cs = headertuple
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
463 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
464 return node, p1, p2, deltabase, cs, flags
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
465
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
466 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
467 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
468
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
469 cg3 streams add support for exchanging treemanifests and revlog
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
470 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
471 separating manifests and files.
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
472 """
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
473 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38896
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38895
diff changeset
474 deltaheadersize = deltaheader.size
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
475 version = '03'
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
476 _grouplistcount = 2 # One list of manifests and one list of files
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
477
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
478 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
479 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
480 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
481
38346
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
482 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38164
diff changeset
483 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29724
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29690
diff changeset
484 for chunkdata in iter(self.filelogheader, {}):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
485 # If we get here, there are directory manifests in the changegroup
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
486 d = chunkdata["filename"]
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
487 repo.ui.debug("adding %s revisions\n" % d)
30339
6cdfb7e15a35 changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents: 30294
diff changeset
488 dirlog = repo.manifestlog._revlog.dirlog(d)
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
489 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
490 if not dirlog.addgroup(deltas, revmap, trp):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
491 raise error.Abort(_("received dir revlog group is empty"))
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
492
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
493 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
494 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
495 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
496 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
497 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
498 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
499 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
500 if len(d) < n:
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
501 d += readexactly(self._fh, n - len(d))
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
502 return d
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
503 return readexactly(self._fh, n)
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
504
39230
b518d495a560 repository: formalize interfaces for revision deltas and requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39229
diff changeset
505 @interfaceutil.implementer(repository.irevisiondeltarequest)
38893
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
506 @attr.s(slots=True, frozen=True)
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
507 class revisiondeltarequest(object):
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
508 node = attr.ib()
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
509 linknode = attr.ib()
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
510 p1node = attr.ib()
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
511 p2node = attr.ib()
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
512 basenode = attr.ib()
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
513 ellipsis = attr.ib(default=False)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
514
39230
b518d495a560 repository: formalize interfaces for revision deltas and requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39229
diff changeset
515 @interfaceutil.implementer(repository.irevisiondelta)
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
516 @attr.s(slots=True, frozen=True)
38893
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
517 class revisiondelta(object):
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
518 node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
519 p1node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
520 p2node = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
521 basenode = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
522 linknode = attr.ib()
23d582caae30 changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38892
diff changeset
523 flags = attr.ib()
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
524 baserevisionsize = attr.ib()
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
525 revision = attr.ib()
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
526 delta = attr.ib()
38883
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38835
diff changeset
527
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
528 def _revisiondeltatochunks(delta, headerfn):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
529 """Serialize a revisiondelta to changegroup chunks."""
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
530
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
531 # The captured revision delta may be encoded as a delta against
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
532 # a base revision or as a full revision. The changegroup format
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
533 # requires that everything on the wire be deltas. So for full
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
534 # revisions, we need to invent a header that says to rewrite
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
535 # data.
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
536
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
537 if delta.delta is not None:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
538 prefix, data = b'', delta.delta
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
539 elif delta.basenode == nullid:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
540 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
541 prefix = mdiff.trivialdiffheader(len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
542 else:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
543 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
544 prefix = mdiff.replacediffheader(delta.baserevisionsize,
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
545 len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
546
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
547 meta = headerfn(delta)
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
548
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
549 yield chunkheader(len(meta) + len(prefix) + len(data))
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
550 yield meta
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
551 if prefix:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
552 yield prefix
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
553 yield data
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
554
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
555 def _sortnodesnormal(store, nodes, reorder):
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
556 """Sort nodes for changegroup generation and turn into revnums."""
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
557 # for generaldelta revlogs, we linearize the revs; this will both be
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
558 # much quicker and generate a much smaller bundle
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
559 if (store._generaldelta and reorder is None) or reorder:
39181
0a934ee94f09 dagop: port revlogdag.linearize() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
560 revs = set(store.rev(n) for n in nodes)
0a934ee94f09 dagop: port revlogdag.linearize() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
561 return dagop.linearize(revs, store.parentrevs)
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
562 else:
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
563 return sorted([store.rev(n) for n in nodes])
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
564
38997
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38996
diff changeset
565 def _sortnodesellipsis(store, nodes, cl, lookup):
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
566 """Sort nodes for changegroup generation and turn into revnums."""
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
567 # Ellipses serving mode.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
568 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
569 # In a perfect world, we'd generate better ellipsis-ified graphs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
570 # for non-changelog revlogs. In practice, we haven't started doing
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
571 # that yet, so the resulting DAGs for the manifestlog and filelogs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
572 # are actually full of bogus parentage on all the ellipsis
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
573 # nodes. This has the side effect that, while the contents are
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
574 # correct, the individual DAGs might be completely out of whack in
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
575 # a case like 882681bc3166 and its ancestors (back about 10
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
576 # revisions or so) in the main hg repo.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
577 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
578 # The one invariant we *know* holds is that the new (potentially
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
579 # bogus) DAG shape will be valid if we order the nodes in the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
580 # order that they're introduced in dramatis personae by the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
581 # changelog, so what we do is we sort the non-changelog histories
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
582 # by the order in which they are used by the changelog.
38997
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38996
diff changeset
583 key = lambda n: cl.rev(lookup(n))
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
584 return [store.rev(n) for n in sorted(nodes, key=key)]
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
585
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
586 def _handlerevisiondeltarequest(store, request, prevnode):
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
587 """Obtain a revisiondelta from a revisiondeltarequest"""
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
588
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
589 node = request.node
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
590 rev = store.rev(node)
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
591
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
592 # Requesting a full revision.
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
593 if request.basenode == nullid:
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
594 baserev = nullrev
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
595 # Requesting an explicit revision.
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
596 elif request.basenode is not None:
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
597 baserev = store.rev(request.basenode)
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
598 # Allowing us to choose.
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
599 else:
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
600 p1, p2 = store.parentrevs(rev)
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
601 dp = store.deltaparent(rev)
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
602
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
603 if dp == nullrev and store.storedeltachains:
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
604 # Avoid sending full revisions when delta parent is null. Pick prev
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
605 # in that case. It's tempting to pick p1 in this case, as p1 will
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
606 # be smaller in the common case. However, computing a delta against
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
607 # p1 may require resolving the raw text of p1, which could be
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
608 # expensive. The revlog caches should have prev cached, meaning
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
609 # less CPU for changegroup generation. There is likely room to add
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
610 # a flag and/or config option to control this behavior.
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
611 baserev = store.rev(prevnode)
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
612 elif dp == nullrev:
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
613 # revlog is configured to use full snapshot for a reason,
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
614 # stick to full snapshot.
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
615 baserev = nullrev
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
616 elif dp not in (p1, p2, store.rev(prevnode)):
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
617 # Pick prev when we can't be sure remote has the base revision.
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
618 baserev = store.rev(prevnode)
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
619 else:
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
620 baserev = dp
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
621
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
622 if baserev != nullrev and not store.candelta(baserev, rev):
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
623 baserev = nullrev
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
624
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
625 revision = None
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
626 delta = None
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
627 baserevisionsize = None
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
628
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
629 if store.iscensored(baserev) or store.iscensored(rev):
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
630 try:
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
631 revision = store.revision(node, raw=True)
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
632 except error.CensoredNodeError as e:
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
633 revision = e.tombstone
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
634
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
635 if baserev != nullrev:
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
636 baserevisionsize = store.rawsize(baserev)
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
637
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
638 elif baserev == nullrev:
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
639 revision = store.revision(node, raw=True)
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
640 else:
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
641 delta = store.revdiff(baserev, rev)
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
642
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
643 extraflags = revlog.REVIDX_ELLIPSIS if request.ellipsis else 0
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
644
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
645 return revisiondelta(
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
646 node=node,
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
647 p1node=request.p1node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
648 p2node=request.p2node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
649 linknode=request.linknode,
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
650 basenode=store.node(baserev),
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
651 flags=store.flags(rev) | extraflags,
39016
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
652 baserevisionsize=baserevisionsize,
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
653 revision=revision,
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39015
diff changeset
654 delta=delta,
38985
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
655 )
d85b0d81112b changegroup: extract _revisiondeltanormal() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38984
diff changeset
656
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
657 def _makenarrowdeltarequest(cl, store, ischangelog, rev, node, linkrev,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
658 linknode, clrevtolocalrev, fullclnodes,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
659 precomputedellipsis):
39004
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
660 linkparents = precomputedellipsis[linkrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
661 def local(clrev):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
662 """Turn a changelog revnum into a local revnum.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
663
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
664 The ellipsis dag is stored as revnums on the changelog,
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
665 but when we're producing ellipsis entries for
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
666 non-changelog revlogs, we need to turn those numbers into
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
667 something local. This does that for us, and during the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
668 changelog sending phase will also expand the stored
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
669 mappings as needed.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
670 """
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
671 if clrev == nullrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
672 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
673
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
674 if ischangelog:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
675 return clrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
676
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
677 # Walk the ellipsis-ized changelog breadth-first looking for a
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
678 # change that has been linked from the current revlog.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
679 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
680 # For a flat manifest revlog only a single step should be necessary
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
681 # as all relevant changelog entries are relevant to the flat
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
682 # manifest.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
683 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
684 # For a filelog or tree manifest dirlog however not every changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
685 # entry will have been relevant, so we need to skip some changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
686 # nodes even after ellipsis-izing.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
687 walk = [clrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
688 while walk:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
689 p = walk[0]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
690 walk = walk[1:]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
691 if p in clrevtolocalrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
692 return clrevtolocalrev[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
693 elif p in fullclnodes:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
694 walk.extend([pp for pp in cl.parentrevs(p)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
695 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
696 elif p in precomputedellipsis:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
697 walk.extend([pp for pp in precomputedellipsis[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
698 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
699 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
700 # In this case, we've got an ellipsis with parents
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
701 # outside the current bundle (likely an
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
702 # incremental pull). We "know" that we can use the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
703 # value of this same revlog at whatever revision
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
704 # is pointed to by linknode. "Know" is in scare
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
705 # quotes because I haven't done enough examination
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
706 # of edge cases to convince myself this is really
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
707 # a fact - it works for all the (admittedly
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
708 # thorough) cases in our testsuite, but I would be
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
709 # somewhat unsurprised to find a case in the wild
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
710 # where this breaks down a bit. That said, I don't
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
711 # know if it would hurt anything.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
712 for i in pycompat.xrange(rev, 0, -1):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
713 if store.linkrev(i) == clrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
714 return i
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
715 # We failed to resolve a parent for this node, so
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
716 # we crash the changegroup construction.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
717 raise error.Abort(
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
718 'unable to resolve parent while packing %r %r'
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
719 ' for changeset %r' % (store.indexfile, rev, clrev))
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
720
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
721 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
722
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
723 if not linkparents or (
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
724 store.parentrevs(rev) == (nullrev, nullrev)):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
725 p1, p2 = nullrev, nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
726 elif len(linkparents) == 1:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
727 p1, = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
728 p2 = nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
729 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
730 p1, p2 = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
731
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
732 p1node, p2node = store.node(p1), store.node(p2)
39004
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
733
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
734 # TODO: try and actually send deltas for ellipsis data blocks
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
735 return revisiondeltarequest(
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
736 node=node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
737 p1node=p1node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
738 p2node=p2node,
39004
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
739 linknode=linknode,
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
740 basenode=nullid,
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
741 ellipsis=True,
39004
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
742 )
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39003
diff changeset
743
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
744 def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev,
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
745 allowreorder,
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
746 units=None,
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
747 ellipses=False, clrevtolocalrev=None, fullclnodes=None,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
748 precomputedellipsis=None):
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
749 """Calculate deltas for a set of revisions.
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
750
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
751 Is a generator of ``revisiondelta`` instances.
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
752
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
753 If units is not None, progress detail will be generated, units specifies
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
754 the type of revlog that is touched (changelog, manifest, etc.).
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
755 """
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
756 if not nodes:
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
757 return
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
758
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
759 # We perform two passes over the revisions whose data we will emit.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
760 #
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
761 # In the first pass, we obtain information about the deltas that will
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
762 # be generated. This involves computing linknodes and adjusting the
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
763 # request to take shallow fetching into account. The end result of
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
764 # this pass is a list of "request" objects stating which deltas
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
765 # to obtain.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
766 #
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
767 # The second pass is simply resolving the requested deltas.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
768
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
769 cl = repo.changelog
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
770
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
771 if ischangelog:
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
772 # Changelog doesn't benefit from reordering revisions. So send
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
773 # out revisions in store order.
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
774 # TODO the API would be cleaner if this were controlled by the
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
775 # store producing the deltas.
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
776 revs = sorted(cl.rev(n) for n in nodes)
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
777 elif ellipses:
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
778 revs = _sortnodesellipsis(store, nodes, cl, lookup)
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
779 else:
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
780 revs = _sortnodesnormal(store, nodes, allowreorder)
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
781
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
782 # In the first pass, collect info about the deltas we'll be
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
783 # generating.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
784 requests = []
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
785
39015
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
786 # Add the parent of the first rev.
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
787 revs.insert(0, store.parentrevs(revs[0])[0])
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
788
39015
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
789 for i in pycompat.xrange(len(revs) - 1):
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
790 prev = revs[i]
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
791 curr = revs[i + 1]
ad9eccedb379 changegroup: minor cleanups to deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39014
diff changeset
792
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
793 node = store.node(curr)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
794 linknode = lookup(node)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
795 p1node, p2node = store.parents(node)
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
796
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
797 if ellipses:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
798 linkrev = cl.rev(linknode)
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
799 clrevtolocalrev[linkrev] = curr
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
800
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
801 # This is a node to send in full, because the changeset it
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
802 # corresponds to was a full changeset.
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
803 if linknode in fullclnodes:
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
804 requests.append(revisiondeltarequest(
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
805 node=node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
806 p1node=p1node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
807 p2node=p2node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
808 linknode=linknode,
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
809 basenode=None,
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
810 ))
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
811
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
812 elif linkrev not in precomputedellipsis:
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
813 pass
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
814 else:
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
815 requests.append(_makenarrowdeltarequest(
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
816 cl, store, ischangelog, curr, node, linkrev, linknode,
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
817 clrevtolocalrev, fullclnodes,
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
818 precomputedellipsis))
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
819 else:
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
820 requests.append(revisiondeltarequest(
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
821 node=node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
822 p1node=p1node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
823 p2node=p2node,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
824 linknode=linknode,
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
825 basenode=store.node(prev) if forcedeltaparentprev else None,
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
826 ))
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
827
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
828 # We expect the first pass to be fast, so we only engage the progress
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
829 # meter for constructing the revision deltas.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
830 progress = None
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
831 if units is not None:
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
832 progress = repo.ui.makeprogress(_('bundling'), unit=units,
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
833 total=len(requests))
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
834
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
835 prevnode = store.node(revs[0])
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
836 for i, request in enumerate(requests):
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
837 if progress:
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
838 progress.update(i + 1)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
839
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
840 delta = _handlerevisiondeltarequest(store, request, prevnode)
39018
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
841
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
842 yield delta
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39017
diff changeset
843
39019
d0d197ab0646 changegroup: capture base node instead of rev in delta request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
844 prevnode = request.node
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
845
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
846 if progress:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
847 progress.complete()
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
848
38902
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
849 class cgpacker(object):
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
850 def __init__(self, repo, filematcher, version, allowreorder,
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
851 builddeltaheader, manifestsend,
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
852 forcedeltaparentprev=False,
38925
8a13855c8dbe changegroup: always use the treemanifest-enabled version of _packmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38910
diff changeset
853 bundlecaps=None, ellipses=False,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
854 shallow=False, ellipsisroots=None, fullnodes=None):
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
855 """Given a source repo, construct a bundler.
32287
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
856
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
857 filematcher is a matcher that matches on files to include in the
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
858 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
859
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
860 allowreorder controls whether reordering of revisions is allowed.
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
861 This value is used when ``bundle.reorder`` is ``auto`` or isn't
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
862 set.
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
863
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
864 forcedeltaparentprev indicates whether delta parents must be against
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
865 the previous revision in a delta group. This should only be used for
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
866 compatibility with changegroup version 1.
38901
23ae0c07a3e1 changegroup: control delta parent behavior via constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38900
diff changeset
867
38897
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
868 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
869 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
870
38898
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38897
diff changeset
871 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38897
diff changeset
872
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
873 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
874
32287
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
875 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
876 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
877 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
878 capabilities to customize the changegroup packer.
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
879
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
880 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
881 need to pack file contents not introduced by the changes being packed.
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
882
38996
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
883 fullnodes is the set of changelog nodes which should not be ellipsis
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
884 nodes. We store this rather than the set of nodes that should be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
885 ellipsis because for very large histories we expect this to be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
886 significantly smaller.
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
887 """
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
888 assert filematcher
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
889 self._filematcher = filematcher
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
890
38895
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38894
diff changeset
891 self.version = version
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
892 self._forcedeltaparentprev = forcedeltaparentprev
38897
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
893 self._builddeltaheader = builddeltaheader
38898
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38897
diff changeset
894 self._manifestsend = manifestsend
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
895 self._ellipses = ellipses
38895
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38894
diff changeset
896
32287
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
897 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
898 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
899 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32268
diff changeset
900 self._bundlecaps = bundlecaps
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
901 self._isshallow = shallow
38996
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
902 self._fullclnodes = fullnodes
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
903
38907
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
904 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
905 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
906
25831
578fc97904da generaldelta: mark experimental reordering option
Matt Mackall <mpm@selenic.com>
parents: 25823
diff changeset
907 # experimental config: bundle.reorder
33183
9f95f0bb343b configitems: register the 'bundle.reorder' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33036
diff changeset
908 reorder = repo.ui.config('bundle', 'reorder')
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
909 if reorder == 'auto':
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
910 self._reorder = allowreorder
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
911 else:
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
912 self._reorder = stringutil.parsebool(reorder)
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
913
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
914 self._repo = repo
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
915
23748
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
916 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
917 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
918 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
919 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
920
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
921 def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
922 """Yield a sequence of changegroup byte chunks."""
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
923
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
924 repo = self._repo
24978
f52560c64953 changegroup: drop _changelog and _manifest properties
Martin von Zweigbergk <martinvonz@google.com>
parents: 24977
diff changeset
925 cl = repo.changelog
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
926
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
927 self._verbosenote(_('uncompressed size of bundle content:\n'))
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
928 size = 0
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
929
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
930 clstate, deltas = self._generatechangelog(cl, clnodes)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
931 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
932 for chunk in _revisiondeltatochunks(delta, self._builddeltaheader):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
933 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
934 yield chunk
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
935
39010
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
936 close = closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
937 size += len(close)
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
938 yield closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
939
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
940 self._verbosenote(_('%8.i (changelog)\n') % size)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
941
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
942 clrevorder = clstate['clrevorder']
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
943 mfs = clstate['mfs']
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
944 changedfiles = clstate['changedfiles']
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
945
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
946 # We need to make sure that the linkrev in the changegroup refers to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
947 # the first changeset that introduced the manifest or file revision.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
948 # The fastpath is usually safer than the slowpath, because the filelogs
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
949 # are walked in revlog order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
950 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
951 # When taking the slowpath with reorder=None and the manifest revlog
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
952 # uses generaldelta, the manifest may be walked in the "wrong" order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
953 # Without 'clrevorder', we would get an incorrect linkrev (see fix in
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
954 # cc0ff93d0c0c).
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
955 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
956 # When taking the fastpath, we are only vulnerable to reordering
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
957 # of the changelog itself. The changelog never uses generaldelta, so
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
958 # it is only reordered when reorder=True. To handle this case, we
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
959 # simply take the slowpath, which already has the 'clrevorder' logic.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
960 # This was also fixed in cc0ff93d0c0c.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
961 fastpathlinkrev = fastpathlinkrev and not self._reorder
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
962 # Treemanifests don't work correctly with fastpathlinkrev
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
963 # either, because we don't discover which directory nodes to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
964 # send along with files. This could probably be fixed.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
965 fastpathlinkrev = fastpathlinkrev and (
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
966 'treemanifest' not in repo.requirements)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
967
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
968 fnodes = {} # needed file nodes
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
969
39011
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39010
diff changeset
970 size = 0
39012
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
971 it = self.generatemanifests(
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
972 commonrevs, clrevorder, fastpathlinkrev, mfs, fnodes, source,
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
973 clstate['clrevtomanifestrev'])
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
974
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
975 for dir, deltas in it:
39012
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
976 if dir:
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
977 assert self.version == b'03'
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
978 chunk = _fileheader(dir)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
979 size += len(chunk)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
980 yield chunk
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
981
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
982 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
983 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
984 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
985 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
986 yield chunk
39012
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
987
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
988 close = closechunk()
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
989 size += len(close)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39011
diff changeset
990 yield close
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
991
39011
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39010
diff changeset
992 self._verbosenote(_('%8.i (manifests)\n') % size)
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39010
diff changeset
993 yield self._manifestsend
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39010
diff changeset
994
38983
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38982
diff changeset
995 mfdicts = None
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38982
diff changeset
996 if self._ellipses and self._isshallow:
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38982
diff changeset
997 mfdicts = [(self._repo.manifestlog[n].read(), lr)
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38982
diff changeset
998 for (n, lr) in mfs.iteritems()]
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
999
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1000 mfs.clear()
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1001 clrevs = set(cl.rev(x) for x in clnodes)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1002
39013
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1003 it = self.generatefiles(changedfiles, commonrevs,
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1004 source, mfdicts, fastpathlinkrev,
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1005 fnodes, clrevs)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1006
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1007 for path, deltas in it:
39013
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1008 h = _fileheader(path)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1009 size = len(h)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1010 yield h
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1011
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1012 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1013 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1014 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1015 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1016 yield chunk
39013
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1017
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1018 close = closechunk()
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1019 size += len(close)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1020 yield close
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1021
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39012
diff changeset
1022 self._verbosenote(_('%8.i %s\n') % (size, path))
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1023
39002
eb8a0139ace3 changegroup: inline _close()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39001
diff changeset
1024 yield closechunk()
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1025
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1026 if clnodes:
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1027 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1028
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1029 def _generatechangelog(self, cl, nodes):
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1030 """Generate data for changelog chunks.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1031
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1032 Returns a 2-tuple of a dict containing state and an iterable of
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1033 byte chunks. The state will not be fully populated until the
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1034 chunk stream has been fully consumed.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1035 """
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1036 clrevorder = {}
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1037 mfs = {} # needed manifests
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1038 mfl = self._repo.manifestlog
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1039 # TODO violates storage abstraction.
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1040 mfrevlog = mfl._revlog
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1041 changedfiles = set()
38998
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38997
diff changeset
1042 clrevtomanifestrev = {}
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1043
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1044 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1045 # manifest nodes.
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1046 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1047 def lookupcl(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1048 c = cl.read(x)
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1049 clrevorder[x] = len(clrevorder)
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1050
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1051 if self._ellipses:
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1052 # Only update mfs if x is going to be sent. Otherwise we
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1053 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1054 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1055 # have sent.
38996
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38995
diff changeset
1056 if (x in self._fullclnodes
38907
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
1057 or cl.rev(x) in self._precomputedellipsis):
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1058 n = c[0]
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1059 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1060 # version.
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1061 mfs.setdefault(n, x)
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1062 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1063 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1064 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1065 # parents)
38998
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38997
diff changeset
1066 clrevtomanifestrev.setdefault(cl.rev(x), mfrevlog.rev(n))
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1067 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1068 # client requested a shallow clone.
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1069 if self._isshallow:
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1070 changedfiles.update(mfl[c[0]].read().keys())
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1071 else:
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1072 changedfiles.update(c[3])
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1073 else:
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1074
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1075 n = c[0]
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1076 # record the first changeset introducing this manifest version
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1077 mfs.setdefault(n, x)
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1078 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1079 # this manifest.
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1080 changedfiles.update(c[3])
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1081
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1082 return x
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1083
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1084 state = {
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1085 'clrevorder': clrevorder,
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1086 'mfs': mfs,
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1087 'changedfiles': changedfiles,
38998
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38997
diff changeset
1088 'clrevtomanifestrev': clrevtomanifestrev,
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1089 }
38890
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38889
diff changeset
1090
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1091 gen = deltagroup(
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1092 self._repo, cl, nodes, True, lookupcl,
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
1093 self._forcedeltaparentprev,
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1094 # Reorder settings are currently ignored for changelog.
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1095 True,
39009
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1096 ellipses=self._ellipses,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1097 units=_('changesets'),
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1098 clrevtolocalrev={},
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1099 fullclnodes=self._fullclnodes,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39008
diff changeset
1100 precomputedellipsis=self._precomputedellipsis)
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1101
38976
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38975
diff changeset
1102 return state, gen
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1103
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1104 def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, mfs,
39001
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1105 fnodes, source, clrevtolocalrev):
34149
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34148
diff changeset
1106 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34148
diff changeset
1107
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34148
diff changeset
1108 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34148
diff changeset
1109 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34148
diff changeset
1110 """
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1111 repo = self._repo
30294
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30268
diff changeset
1112 mfl = repo.manifestlog
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30268
diff changeset
1113 dirlog = mfl._revlog.dirlog
28232
829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents: 28231
diff changeset
1114 tmfnodes = {'': mfs}
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1115
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1116 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1117 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1118 # Returns the linkrev node (collected in lookupcl).
35012
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1119 def makelookupmflinknode(dir, nodes):
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1120 if fastpathlinkrev:
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1121 assert not dir
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1122 return mfs.__getitem__
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1123
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1124 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1125 """Callback for looking up the linknode for manifests.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1126
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1127 Returns the linkrev node for the specified manifest.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1128
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1129 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1130
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1131 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1132 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1133 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1134 to send
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1135
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1136 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1137 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1138 treemanifests to send.
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1139 """
35012
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1140 clnode = nodes[x]
30294
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30268
diff changeset
1141 mdata = mfl.get(dir, x).readfast(shallow=True)
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1142 for p, n, fl in mdata.iterentries():
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1143 if fl == 't': # subdirectory manifest
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1144 subdir = dir + p + '/'
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1145 tmfclnodes = tmfnodes.setdefault(subdir, {})
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1146 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1147 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1148 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1149 else:
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1150 f = dir + p
28240
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1151 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1152 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1153 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1154 fclnodes[n] = clnode
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1155 return clnode
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1156 return lookupmflinknode
19206
6308896b1d4a bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents: 19204
diff changeset
1157
28232
829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents: 28231
diff changeset
1158 while tmfnodes:
35012
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1159 dir, nodes = tmfnodes.popitem()
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
1160 store = dirlog(dir)
39007
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1161
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1162 if not self._filematcher.visitdir(store._dir[:-1] or '.'):
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1163 prunednodes = []
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1164 else:
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1165 frev, flr = store.rev, store.linkrev
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1166 prunednodes = [n for n in nodes
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1167 if flr(frev(n)) not in commonrevs]
39005
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39004
diff changeset
1168
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39004
diff changeset
1169 if dir and not prunednodes:
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39004
diff changeset
1170 continue
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39004
diff changeset
1171
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39004
diff changeset
1172 lookupfn = makelookupmflinknode(dir, nodes)
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
1173
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1174 deltas = deltagroup(
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1175 self._repo, store, prunednodes, False, lookupfn,
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1176 self._forcedeltaparentprev, self._reorder,
39008
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1177 ellipses=self._ellipses,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1178 units=_('manifests'),
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1179 clrevtolocalrev=clrevtolocalrev,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1180 fullclnodes=self._fullclnodes,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1181 precomputedellipsis=self._precomputedellipsis)
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1182
39014
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1183 yield dir, deltas
39010
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
1184
24897
5c35a6040352 changegroup: document that 'source' parameter exists for extensions
Martin von Zweigbergk <martinvonz@google.com>
parents: 24896
diff changeset
1185 # The 'source' parameter is useful for extensions
38999
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1186 def generatefiles(self, changedfiles, commonrevs, source,
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1187 mfdicts, fastpathlinkrev, fnodes, clrevs):
38889
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1188 changedfiles = list(filter(self._filematcher, changedfiles))
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1189
38999
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1190 if not fastpathlinkrev:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1191 def normallinknodes(unused, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1192 return fnodes.get(fname, {})
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1193 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1194 cln = self._repo.changelog.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1195
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1196 def normallinknodes(store, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1197 flinkrev = store.linkrev
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1198 fnode = store.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1199 revs = ((r, flinkrev(r)) for r in store)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1200 return dict((fnode(r), cln(lr))
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1201 for r, lr in revs if lr in clrevs)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1202
39001
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1203 clrevtolocalrev = {}
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1204
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1205 if self._isshallow:
38889
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1206 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1207 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1208 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1209 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1210 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1211
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1212 # Defining this function has a side-effect of overriding the
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1213 # function of the same name that was passed in as an argument.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1214 # TODO have caller pass in appropriate function.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1215 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1216 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1217 try:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1218 fnode = c.filenode(fname)
39001
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1219 clrevtolocalrev[c.rev()] = flog.rev(fnode)
38889
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1220 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1221 pass
38999
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1222 links = normallinknodes(flog, fname)
38889
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1223 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1224 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1225 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1226 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1227 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1228 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1229 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1230 return links
38999
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1231 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38998
diff changeset
1232 linknodes = normallinknodes
38889
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38888
diff changeset
1233
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1234 repo = self._repo
38410
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38382
diff changeset
1235 progress = repo.ui.makeprogress(_('bundling'), unit=_('files'),
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38382
diff changeset
1236 total=len(changedfiles))
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1237 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1238 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1239 if not filerevlog:
37339
5859800edfc5 changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37132
diff changeset
1240 raise error.Abort(_("empty or missing file data for %s") %
5859800edfc5 changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37132
diff changeset
1241 fname)
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1242
39001
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1243 clrevtolocalrev.clear()
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39000
diff changeset
1244
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1245 linkrevnodes = linknodes(filerevlog, fname)
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1246 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1247 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1248 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1249 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1250
39007
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1251 frev, flr = filerevlog.rev, filerevlog.linkrev
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1252 filenodes = [n for n in linkrevnodes
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1253 if flr(frev(n)) not in commonrevs]
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39006
diff changeset
1254
39020
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1255 if not filenodes:
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1256 continue
38982
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38981
diff changeset
1257
39020
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1258 progress.update(i + 1, item=fname)
39008
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39007
diff changeset
1259
39020
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1260 deltas = deltagroup(
39229
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1261 self._repo, filerevlog, filenodes, False, lookupfilelog,
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39181
diff changeset
1262 self._forcedeltaparentprev, self._reorder,
39020
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1263 ellipses=self._ellipses,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1264 clrevtolocalrev=clrevtolocalrev,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1265 fullclnodes=self._fullclnodes,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1266 precomputedellipsis=self._precomputedellipsis)
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1267
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1268 yield fname, deltas
39010
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39009
diff changeset
1269
38410
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38382
diff changeset
1270 progress.complete()
19200
4cfdec944edf bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents: 19199
diff changeset
1271
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1272 def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1273 shallow=False, ellipsisroots=None, fullnodes=None):
38897
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1274 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1275 d.node, d.p1node, d.p2node, d.linknode)
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1276
38902
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1277 return cgpacker(repo, filematcher, b'01',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1278 allowreorder=None,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1279 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1280 manifestsend=b'',
39017
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39016
diff changeset
1281 forcedeltaparentprev=True,
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1282 bundlecaps=bundlecaps,
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1283 ellipses=ellipses,
38907
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
1284 shallow=shallow,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1285 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1286 fullnodes=fullnodes)
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1287
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1288 def _makecg2packer(repo, filematcher, bundlecaps, ellipses=False,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1289 shallow=False, ellipsisroots=None, fullnodes=None):
38897
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1290 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1291 d.node, d.p1node, d.p2node, d.basenode, d.linknode)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1292
38900
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
1293 # Since generaldelta is directly supported by cg2, reordering
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
1294 # generally doesn't help, so we disable it by default (treating
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38899
diff changeset
1295 # bundle.reorder=auto just like bundle.reorder=False).
38902
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1296 return cgpacker(repo, filematcher, b'02',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1297 allowreorder=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1298 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1299 manifestsend=b'',
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1300 bundlecaps=bundlecaps,
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1301 ellipses=ellipses,
38907
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
1302 shallow=shallow,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1303 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1304 fullnodes=fullnodes)
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1305
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1306 def _makecg3packer(repo, filematcher, bundlecaps, ellipses=False,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1307 shallow=False, ellipsisroots=None, fullnodes=None):
38897
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1308 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1309 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38896
diff changeset
1310
38902
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1311 return cgpacker(repo, filematcher, b'03',
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1312 allowreorder=False,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1313 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38901
diff changeset
1314 manifestsend=closechunk(),
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1315 bundlecaps=bundlecaps,
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1316 ellipses=ellipses,
38907
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38906
diff changeset
1317 shallow=shallow,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1318 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1319 fullnodes=fullnodes)
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1320
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1321 _packermap = {'01': (_makecg1packer, cg1unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1322 # cg2 adds support for exchanging generaldelta
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1323 '02': (_makecg2packer, cg2unpacker),
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
1324 # cg3 adds support for exchanging revlog flags and treemanifests
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1325 '03': (_makecg3packer, cg3unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1326 }
23168
a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22971
diff changeset
1327
30627
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30626
diff changeset
1328 def allsupportedversions(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1329 versions = set(_packermap.keys())
30627
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30626
diff changeset
1330 if not (repo.ui.configbool('experimental', 'changegroup3') or
30628
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30627
diff changeset
1331 repo.ui.configbool('experimental', 'treemanifest') or
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30627
diff changeset
1332 'treemanifest' in repo.requirements):
30626
438532c99b54 changegroup: simplify 'allsupportedversions' logic
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30589
diff changeset
1333 versions.discard('03')
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1334 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1335
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1336 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1337 def supportedincomingversions(repo):
30628
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30627
diff changeset
1338 return allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1339
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1340 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1341 def supportedoutgoingversions(repo):
30627
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30626
diff changeset
1342 versions = allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1343 if 'treemanifest' in repo.requirements:
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1344 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1345 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1346 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1347 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1348 # support versions 01 and 02.
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1349 versions.discard('01')
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1350 versions.discard('02')
38835
a232e6744ba3 narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents: 38806
diff changeset
1351 if repository.NARROW_REQUIREMENT in repo.requirements:
36465
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36464
diff changeset
1352 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36464
diff changeset
1353 # support that for stripping and unbundling to work.
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36464
diff changeset
1354 versions.discard('01')
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36464
diff changeset
1355 versions.discard('02')
37132
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1356 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1357 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1358 # mark LFS entries with REVIDX_EXTSTORED.
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1359 versions.discard('01')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1360 versions.discard('02')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37084
diff changeset
1361
27752
29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config
Martin von Zweigbergk <martinvonz@google.com>
parents: 27751
diff changeset
1362 return versions
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1363
34144
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1364 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1365 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1366 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1367 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1368
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1369 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1370 # Finds the smallest version that it's safe to assume clients of the repo
27931
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1371 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1372 # support changegroup 02.
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1373 versions = supportedoutgoingversions(repo)
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1374 if 'generaldelta' in repo.requirements:
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1375 versions.discard('01')
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1376 assert versions
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1377 return min(versions)
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1378
38904
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38903
diff changeset
1379 def getbundler(version, repo, bundlecaps=None, filematcher=None,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1380 ellipses=False, shallow=False, ellipsisroots=None,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1381 fullnodes=None):
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1382 assert version in supportedoutgoingversions(repo)
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1383
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1384 if filematcher is None:
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1385 filematcher = matchmod.alwaysmatcher(repo.root, '')
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1386
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1387 if version == '01' and not filematcher.always():
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1388 raise error.ProgrammingError('version 01 changegroups do not support '
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1389 'sparse file matchers')
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1390
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1391 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1392 raise error.Abort(
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1393 _('ellipsis nodes require at least cg3 on client and server, '
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1394 'but negotiated version %s') % version)
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1395
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1396 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1397 # filter those out.
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1398 filematcher = matchmod.intersectmatchers(repo.narrowmatch(),
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1399 filematcher)
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1400
38894
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38893
diff changeset
1401 fn = _packermap[version][0]
38908
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38907
diff changeset
1402 return fn(repo, filematcher, bundlecaps, ellipses=ellipses,
38909
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1403 shallow=shallow, ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38908
diff changeset
1404 fullnodes=fullnodes)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1405
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1406 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1407 return _packermap[version][1](fh, alg, extras=extras)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1408
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1409 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1410 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1411 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1412 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1413 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1414 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1415 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1416
34096
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1417 def makechangegroup(repo, outgoing, version, source, fastpath=False,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1418 bundlecaps=None):
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1419 cgstream = makestream(repo, outgoing, version, source,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1420 fastpath=fastpath, bundlecaps=bundlecaps)
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1421 return getunbundler(version, util.chunkbuffer(cgstream), None,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1422 {'clcount': len(outgoing.missing) })
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34091
diff changeset
1423
34103
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
1424 def makestream(repo, outgoing, version, source, fastpath=False,
38794
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1425 bundlecaps=None, filematcher=None):
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1426 bundler = getbundler(version, repo, bundlecaps=bundlecaps,
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38783
diff changeset
1427 filematcher=filematcher)
34103
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34101
diff changeset
1428
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1429 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1430 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1431 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1432 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1433 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1434 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1435 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1436 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1437 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1438 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1439
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1440 repo.hook('preoutgoing', throw=True, source=source)
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1441 _changegroupinfo(repo, csets, source)
23177
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1442 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1443
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1444 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1445 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1446 files = 0
38382
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38373
diff changeset
1447 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38373
diff changeset
1448 total=expectedfiles)
29724
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29690
diff changeset
1449 for chunkdata in iter(source.filelogheader, {}):
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1450 files += 1
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1451 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1452 repo.ui.debug("adding %s revisions\n" % f)
38382
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38373
diff changeset
1453 progress.increment()
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1454 fl = repo.file(f)
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1455 o = len(fl)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 23897
diff changeset
1456 try:
34291
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
1457 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34257
diff changeset
1458 if not fl.addgroup(deltas, revmap, trp):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1459 raise error.Abort(_("received file revlog group is empty"))
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25624
diff changeset
1460 except error.CensoredBaseError as e:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1461 raise error.Abort(_("received delta base is censored: %s") % e)
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1462 revisions += len(fl) - o
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1463 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1464 needs = needfiles[f]
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38556
diff changeset
1465 for new in pycompat.xrange(o, len(fl)):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1466 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1467 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1468 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1469 else:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1470 raise error.Abort(
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1471 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1472 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1473 del needfiles[f]
38382
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38373
diff changeset
1474 progress.complete()
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1475
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1476 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1477 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1478 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1479 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1480 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1481 except error.LookupError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1482 raise error.Abort(
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1483 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1484 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1485
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1486 return revisions, files