Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 42823:268662aac075

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
interfaces: create a new folder for interfaces and move repository.py in it I was trying to understand current interfaces and write new ones and I realized we need to improve how current interfaces are organised. This creates a dedicated folder for defining interfaces and move `repository.py` which defines all the current interfaces inside it. Differential Revision: https://phab.mercurial-scm.org/D6741
author Pulkit Goyal <pulkit@yandex-team.ru>
date Sun, 18 Aug 2019 00:45:33 +0300
parents 27d6956d386b
children d7304434390f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
4 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9437
diff changeset
6 # GNU General Public License version 2 or any later version.
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3859
diff changeset
7
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
9
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
10 import os
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
11 import struct
20933
d3775db748a0 localrepo: move the addchangegroup method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20932
diff changeset
12 import weakref
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
13
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
14 from .i18n import _
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
15 from .node import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
16 hex,
38922
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38874
diff changeset
17 nullid,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
18 nullrev,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
19 short,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
20 )
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
21
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
22 from . import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
23 error,
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
24 match as matchmod,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
25 mdiff,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
26 phases,
30945
82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30743
diff changeset
27 pycompat,
42823
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
28 util,
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
29 )
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
30
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
31 from .interfaces import (
38874
a232e6744ba3 narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents: 38846
diff changeset
32 repository,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
33 )
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
34
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
35 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
36 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
37 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
38
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
39 LFS_REQUIREMENT = 'lfs'
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
40
35754
fb0be099063f util: move 'readexactly' in the util module
Boris Feld <boris.feld@octobus.net>
parents: 35051
diff changeset
41 readexactly = util.readexactly
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
42
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
43 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
44 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
45 d = readexactly(stream, 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
46 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
47 if l <= 4:
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
48 if l:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
49 raise error.Abort(_("invalid chunk length %d") % l)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
50 return ""
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
51 return readexactly(stream, l - 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
52
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
53 def chunkheader(length):
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
54 """return a changegroup chunk header (string)"""
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
55 return struct.pack(">l", length + 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
56
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
57 def closechunk():
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
58 """return a changegroup chunk header (string) for a zero-length chunk"""
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
59 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
60
39019
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
61 def _fileheader(path):
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
62 """Obtain a changegroup chunk header for a named path."""
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
63 return chunkheader(len(path)) + path
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
64
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
65 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
66 """Write chunks to a file and return its filename.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
67
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
68 The stream is assumed to be a bundle file.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
69 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
70 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
71 """
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
72 fh = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
73 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
74 try:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
75 if filename:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
76 if vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
77 fh = vfs.open(filename, "wb")
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
78 else:
30212
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
79 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
80 # small (4k is common on Linux).
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
81 fh = open(filename, "wb", 131072)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
82 else:
38197
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37339
diff changeset
83 fd, filename = pycompat.mkstemp(prefix="hg-bundle-", suffix=".hg")
36843
5bc7ff103081 py3: use r'' instead of sysstr('') to get around code transformer
Yuya Nishihara <yuya@tcha.org>
parents: 36768
diff changeset
84 fh = os.fdopen(fd, r"wb")
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
85 cleanup = filename
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
86 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
87 fh.write(c)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
88 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
89 return filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
90 finally:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
91 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
92 fh.close()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
93 if cleanup is not None:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
94 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
95 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
96 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
97 os.unlink(cleanup)
3660
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
98
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
99 class cg1unpacker(object):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
100 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
101
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
102 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
103 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
104 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
105
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
106 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
107 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
108 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
109 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
110
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
111 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
112 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
113
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
114 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
115 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
116 """
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
117 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
118 deltaheadersize = deltaheader.size
23896
becfecaf9087 changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents: 23895
diff changeset
119 version = '01'
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
120 _grouplistcount = 1 # One list of files after the manifests
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
121
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
122 def __init__(self, fh, alg, extras=None):
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
123 if alg is None:
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
124 alg = 'UN'
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
125 if alg not in util.compengines.supportedbundletypes:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
126 raise error.Abort(_('unknown stream compression type: %s')
26266
1e042e31bd0c changegroup: move all compressions utilities in util
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25921
diff changeset
127 % alg)
26392
127b59787fd5 changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26272
diff changeset
128 if alg == 'BZ':
127b59787fd5 changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26272
diff changeset
129 alg = '_truncatedBZ'
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
130
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
131 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
132 self._stream = compengine.decompressorreader(fh)
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
133 self._type = alg
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
134 self.extras = extras or {}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
135 self.callback = None
26706
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
136
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
137 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
138 # be used by bundlerepo, but it's a little hard to tell.
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
139 def compressed(self):
30594
182cacaa4c32 cg1packer: fix `compressed` method
Stanislau Hlebik <stash@fb.com>
parents: 30364
diff changeset
140 return self._type is not None and self._type != 'UN'
12043
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
141 def read(self, l):
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
142 return self._stream.read(l)
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
143 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
144 return self._stream.seek(pos)
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
145 def tell(self):
12332
680fe77ab5b8 bundlerepo: use bundle objects everywhere
Matt Mackall <mpm@selenic.com>
parents: 12330
diff changeset
146 return self._stream.tell()
12347
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
147 def close(self):
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
148 return self._stream.close()
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
149
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
150 def _chunklength(self):
13459
acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34
Jim Hague <jim.hague@acm.org>
parents: 13458
diff changeset
151 d = readexactly(self._stream, 4)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
152 l = struct.unpack(">l", d)[0]
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
153 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
154 if l:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
155 raise error.Abort(_("invalid chunk length %d") % l)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
156 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
157 if self.callback:
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
158 self.callback()
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
159 return l - 4
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
160
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
161 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
162 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
163 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
164
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
165 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
166 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
167 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
168
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
169 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
170 """return the header of the filelogs chunk, v10 only has the filename"""
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
171 l = self._chunklength()
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
172 if not l:
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
173 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
174 fname = readexactly(self._stream, l)
20675
f8d50add83e1 changegroup: move from dict() construction to {} literals
Augie Fackler <raf@durin42.com>
parents: 19708
diff changeset
175 return {'filename': fname}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
176
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
177 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
178 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
179 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
180 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
181 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
182 deltabase = prevnode
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
183 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
184 return node, p1, p2, deltabase, cs, flags
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
185
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
186 def deltachunk(self, prevnode):
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
187 l = self._chunklength()
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
188 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
189 return {}
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
190 headerdata = readexactly(self._stream, self.deltaheadersize)
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
191 header = self.deltaheader.unpack(headerdata)
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
192 delta = readexactly(self._stream, l - self.deltaheadersize)
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
193 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
34301
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
194 return (node, p1, p2, cs, deltabase, delta, flags)
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
195
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
196 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
197 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
198
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
199 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
200 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
201 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
202 """
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
203 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
204 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
205 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
206 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
207 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
208 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
209 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
210 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
211 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
212 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
213 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
214
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
215 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
216 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
217 noentries = True
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
218 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
219 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
220 if not chunk:
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
221 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
222 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
223 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
224 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
225 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
226 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
227 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
228 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
229 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
230 parts += 1
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
231 break
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
232 noentries = False
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
233 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
234 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
235 while pos < len(chunk):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
236 next = pos + 2**20
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
237 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
238 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
239 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
240
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
241 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
242 self.callback = prog.increment
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
243 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
244 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
245 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
246 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
247 self.manifestheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
248 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
249 repo.manifestlog.getstorage(b'').addgroup(deltas, revmap, trp)
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
250 prog.complete()
28360
11287888ce4b changegroup: exclude submanifests from manifest progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 28281
diff changeset
251 self.callback = None
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
252
33308
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
253 def apply(self, repo, tr, srctype, url, targetphase=phases.draft,
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
254 expectedtotal=None):
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
255 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
256 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
257 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
258
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
259 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
260 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
261 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
262 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
263 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
264 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
265 repo = repo.unfiltered()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
266 def csmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
267 repo.ui.debug("add changeset %s\n" % short(x))
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
268 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
269
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
270 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
271 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
272
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
273 changesets = files = revisions = 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
274
26880
fa7f8b686633 changegroup: fix the scope of a try finally
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26859
diff changeset
275 try:
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
276 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
277 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
278 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
279 # in this function.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
280 srctype = tr.hookargs.setdefault('source', srctype)
41387
876494fd967d cleanup: delete lots of unused local variables
Martin von Zweigbergk <martinvonz@google.com>
parents: 40705
diff changeset
281 tr.hookargs.setdefault('url', url)
33697
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
282 repo.hook('prechangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
283 throw=True, **pycompat.strkwargs(tr.hookargs))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
284
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
285 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
286 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
287 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
288 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
289 oldheads = set(cl.heads())
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
290
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
291 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
292 # pull off the changeset group
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
293 repo.ui.status(_("adding changesets\n"))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
294 clstart = len(cl)
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
295 progress = repo.ui.makeprogress(_('changesets'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
296 total=expectedtotal)
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
297 self.callback = progress.increment
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
298
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
299 efiles = set()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
300 def onchangelog(cl, node):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
301 efiles.update(cl.readfiles(node))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
302
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
303 self.changelogheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
304 deltas = self.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
305 cgnodes = cl.addgroup(deltas, csmap, trp, addrevisioncb=onchangelog)
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
306 efiles = len(efiles)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
307
33308
248d5890c80a changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33227
diff changeset
308 if not cgnodes:
39686
9b07ee0a1054 changegroup: improve the devel-warn to specify changelog was empty
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39342
diff changeset
309 repo.ui.develwarn('applied empty changelog from changegroup',
34734
3572b2031cec devel-warn: add 'warn-' to 'devel.empty-changegroup' config
Boris Feld <boris.feld@octobus.net>
parents: 34301
diff changeset
310 config='warn-empty-changegroup')
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
311 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
312 changesets = clend - clstart
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
313 progress.complete()
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
314 self.callback = None
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
315
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
316 # pull off the manifest group
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
317 repo.ui.status(_("adding manifests\n"))
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
318 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
319 # changesets.
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
320 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'),
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
321 total=changesets)
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
322 self._unpackmanifests(repo, revmap, trp, progress)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
323
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
324 needfiles = {}
33227
86c9aa1d598f configitems: register the 'server.validate' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33185
diff changeset
325 if repo.ui.configbool('server', 'validate'):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
326 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
327 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
328 # validate incoming csets have their manifests
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
329 for cset in pycompat.xrange(clstart, clend):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
330 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
331 mfest = ml[mfnode].readdelta()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
332 # store file cgnodes we must see
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
333 for f, n in mfest.iteritems():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
334 needfiles.setdefault(f, set()).add(n)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
335
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
336 # process the files
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
337 repo.ui.status(_("adding file changes\n"))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
338 newrevs, newfiles = _addchangegroupfiles(
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
339 repo, self, revmap, trp, efiles, needfiles)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
340 revisions += newrevs
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
341 files += newfiles
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
342
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
343 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
344 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
345 heads = cl.heads()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
346 deltaheads = len(heads) - len(oldheads)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
347 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
348 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
349 deltaheads -= 1
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
350 htext = ""
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
351 if deltaheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
352 htext = _(" (%+d heads)") % deltaheads
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
353
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
354 repo.ui.status(_("added %d changesets"
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
355 " with %d changes to %d files%s\n")
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
356 % (changesets, revisions, files, htext))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
357 repo.invalidatevolatilesets()
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
358
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
359 if changesets > 0:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
360 if 'node' not in tr.hookargs:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
361 tr.hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
362 tr.hookargs['node_last'] = hex(cl.node(clend - 1))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
363 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
364 else:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
365 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
366 hookargs['node'] = hex(cl.node(clstart))
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
367 hookargs['node_last'] = hex(cl.node(clend - 1))
33697
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
368 repo.hook('pretxnchangegroup',
38fc45721334 changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents: 33461
diff changeset
369 throw=True, **pycompat.strkwargs(hookargs))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
370
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
371 added = [cl.node(r) for r in pycompat.xrange(clstart, clend)]
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
372 phaseall = None
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
373 if srctype in ('push', 'serve'):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
374 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
375 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
376 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
377 #
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
378 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
379 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
380 if repo.publishing():
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
381 targetphase = phaseall = phases.public
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
382 else:
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
383 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
384
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
385 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
386 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
387 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
388 # ignored.
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
389 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
390 if added:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
391 phases.registernew(repo, tr, targetphase, added)
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
392 if phaseall is not None:
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
393 phases.advanceboundary(repo, tr, phaseall, cgnodes)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
394
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
395 if changesets > 0:
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
396
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
397 def runhooks():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
398 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
399 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
400 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
401 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
402 return
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
403
33738
672ad4f3bb84 changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents: 33697
diff changeset
404 repo.hook("changegroup", **pycompat.strkwargs(hookargs))
27867
7ced54ebf972 with: use context manager for transaction in changegroup apply
Bryan O'Sullivan <bryano@fb.com>
parents: 27754
diff changeset
405
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
406 for n in added:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
407 args = hookargs.copy()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
408 args['node'] = hex(n)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
409 del args['node_last']
33738
672ad4f3bb84 changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents: 33697
diff changeset
410 repo.hook("incoming", **pycompat.strkwargs(args))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
411
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
412 newheads = [h for h in repo.heads()
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
413 if h not in oldheads]
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
414 repo.ui.log("incoming",
36760
2a5024109490 py3: fix int formatting of "incoming changes" log
Yuya Nishihara <yuya@tcha.org>
parents: 36495
diff changeset
415 "%d incoming changes - new heads: %s\n",
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
416 len(added),
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
417 ', '.join([hex(c[:6]) for c in newheads]))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
418
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
419 tr.addpostclose('changegroup-runhooks-%020i' % clstart,
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
420 lambda tr: repo._afterlock(runhooks))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
421 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
422 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
423 # never return 0 here:
32888
b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32887
diff changeset
424 if deltaheads < 0:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
425 ret = deltaheads - 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
426 else:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
427 ret = deltaheads + 1
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33456
diff changeset
428 return ret
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
429
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
430 def deltaiter(self):
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
431 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
432 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
433
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
434 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
435 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
436 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
437 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34301
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
438 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
439 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
440 chain = chunkdata[0]
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
441
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
442 class cg2unpacker(cg1unpacker):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
443 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
444
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
445 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
446 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
447 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
448 """
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
449 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
450 deltaheadersize = deltaheader.size
23896
becfecaf9087 changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents: 23895
diff changeset
451 version = '02'
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
452
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
453 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
454 node, p1, p2, deltabase, cs = headertuple
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
455 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
456 return node, p1, p2, deltabase, cs, flags
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
457
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
458 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
459 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
460
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
461 cg3 streams add support for exchanging treemanifests and revlog
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
462 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
463 separating manifests and files.
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
464 """
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
465 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
466 deltaheadersize = deltaheader.size
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
467 version = '03'
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
468 _grouplistcount = 2 # One list of manifests and one list of files
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
469
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
470 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
471 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
472 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
473
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
474 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
475 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
476 for chunkdata in iter(self.filelogheader, {}):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
477 # If we get here, there are directory manifests in the changegroup
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
478 d = chunkdata["filename"]
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
479 repo.ui.debug("adding %s revisions\n" % d)
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
480 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
481 if not repo.manifestlog.getstorage(d).addgroup(deltas, revmap, trp):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
482 raise error.Abort(_("received dir revlog group is empty"))
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
483
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
484 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
485 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
486 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
487 self._fh = fh
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
488 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
489 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
490 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
491 if len(d) < n:
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
492 d += readexactly(self._fh, n - len(d))
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
493 return d
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
494 return readexactly(self._fh, n)
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
495
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
496 def _revisiondeltatochunks(delta, headerfn):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
497 """Serialize a revisiondelta to changegroup chunks."""
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
498
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
499 # The captured revision delta may be encoded as a delta against
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
500 # a base revision or as a full revision. The changegroup format
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
501 # requires that everything on the wire be deltas. So for full
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
502 # revisions, we need to invent a header that says to rewrite
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
503 # data.
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
504
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
505 if delta.delta is not None:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
506 prefix, data = b'', delta.delta
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
507 elif delta.basenode == nullid:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
508 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
509 prefix = mdiff.trivialdiffheader(len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
510 else:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
511 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
512 prefix = mdiff.replacediffheader(delta.baserevisionsize,
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
513 len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
514
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
515 meta = headerfn(delta)
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
516
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
517 yield chunkheader(len(meta) + len(prefix) + len(data))
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
518 yield meta
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
519 if prefix:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
520 yield prefix
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
521 yield data
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
522
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
523 def _sortnodesellipsis(store, nodes, cl, lookup):
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
524 """Sort nodes for changegroup generation."""
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
525 # Ellipses serving mode.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
526 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
527 # In a perfect world, we'd generate better ellipsis-ified graphs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
528 # for non-changelog revlogs. In practice, we haven't started doing
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
529 # that yet, so the resulting DAGs for the manifestlog and filelogs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
530 # are actually full of bogus parentage on all the ellipsis
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
531 # nodes. This has the side effect that, while the contents are
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
532 # correct, the individual DAGs might be completely out of whack in
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
533 # a case like 882681bc3166 and its ancestors (back about 10
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
534 # revisions or so) in the main hg repo.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
535 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
536 # The one invariant we *know* holds is that the new (potentially
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
537 # bogus) DAG shape will be valid if we order the nodes in the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
538 # order that they're introduced in dramatis personae by the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
539 # changelog, so what we do is we sort the non-changelog histories
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
540 # by the order in which they are used by the changelog.
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
541 key = lambda n: cl.rev(lookup(n))
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
542 return sorted(nodes, key=key)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
543
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
544 def _resolvenarrowrevisioninfo(cl, store, ischangelog, rev, linkrev,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
545 linknode, clrevtolocalrev, fullclnodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
546 precomputedellipsis):
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
547 linkparents = precomputedellipsis[linkrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
548 def local(clrev):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
549 """Turn a changelog revnum into a local revnum.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
550
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
551 The ellipsis dag is stored as revnums on the changelog,
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
552 but when we're producing ellipsis entries for
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
553 non-changelog revlogs, we need to turn those numbers into
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
554 something local. This does that for us, and during the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
555 changelog sending phase will also expand the stored
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
556 mappings as needed.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
557 """
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
558 if clrev == nullrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
559 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
560
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
561 if ischangelog:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
562 return clrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
563
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
564 # Walk the ellipsis-ized changelog breadth-first looking for a
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
565 # change that has been linked from the current revlog.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
566 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
567 # For a flat manifest revlog only a single step should be necessary
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
568 # as all relevant changelog entries are relevant to the flat
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
569 # manifest.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
570 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
571 # For a filelog or tree manifest dirlog however not every changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
572 # entry will have been relevant, so we need to skip some changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
573 # nodes even after ellipsis-izing.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
574 walk = [clrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
575 while walk:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
576 p = walk[0]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
577 walk = walk[1:]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
578 if p in clrevtolocalrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
579 return clrevtolocalrev[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
580 elif p in fullclnodes:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
581 walk.extend([pp for pp in cl.parentrevs(p)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
582 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
583 elif p in precomputedellipsis:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
584 walk.extend([pp for pp in precomputedellipsis[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
585 if pp != nullrev])
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
586 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
587 # In this case, we've got an ellipsis with parents
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
588 # outside the current bundle (likely an
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
589 # incremental pull). We "know" that we can use the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
590 # value of this same revlog at whatever revision
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
591 # is pointed to by linknode. "Know" is in scare
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
592 # quotes because I haven't done enough examination
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
593 # of edge cases to convince myself this is really
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
594 # a fact - it works for all the (admittedly
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
595 # thorough) cases in our testsuite, but I would be
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
596 # somewhat unsurprised to find a case in the wild
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
597 # where this breaks down a bit. That said, I don't
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
598 # know if it would hurt anything.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
599 for i in pycompat.xrange(rev, 0, -1):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
600 if store.linkrev(i) == clrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
601 return i
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
602 # We failed to resolve a parent for this node, so
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
603 # we crash the changegroup construction.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
604 raise error.Abort(
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
605 'unable to resolve parent while packing %r %r'
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
606 ' for changeset %r' % (store.indexfile, rev, clrev))
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
607
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
608 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
609
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
610 if not linkparents or (
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
611 store.parentrevs(rev) == (nullrev, nullrev)):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
612 p1, p2 = nullrev, nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
613 elif len(linkparents) == 1:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
614 p1, = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
615 p2 = nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
616 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
617 p1, p2 = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
618
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
619 p1node, p2node = store.node(p1), store.node(p2)
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
620
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
621 return p1node, p2node, linknode
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
622
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
623 def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev,
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
624 topic=None,
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
625 ellipses=False, clrevtolocalrev=None, fullclnodes=None,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
626 precomputedellipsis=None):
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
627 """Calculate deltas for a set of revisions.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
628
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
629 Is a generator of ``revisiondelta`` instances.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
630
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
631 If topic is not None, progress detail will be generated using this
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
632 topic name (e.g. changesets, manifests, etc).
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
633 """
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
634 if not nodes:
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
635 return
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
636
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
637 cl = repo.changelog
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
638
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
639 if ischangelog:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
640 # `hg log` shows changesets in storage order. To preserve order
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
641 # across clones, send out changesets in storage order.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
642 nodesorder = 'storage'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
643 elif ellipses:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
644 nodes = _sortnodesellipsis(store, nodes, cl, lookup)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
645 nodesorder = 'nodes'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
646 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
647 nodesorder = None
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
648
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
649 # Perform ellipses filtering and revision massaging. We do this before
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
650 # emitrevisions() because a) filtering out revisions creates less work
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
651 # for emitrevisions() b) dropping revisions would break emitrevisions()'s
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
652 # assumptions about delta choices and we would possibly send a delta
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
653 # referencing a missing base revision.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
654 #
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
655 # Also, calling lookup() has side-effects with regards to populating
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
656 # data structures. If we don't call lookup() for each node or if we call
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
657 # lookup() after the first pass through each node, things can break -
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
658 # possibly intermittently depending on the python hash seed! For that
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
659 # reason, we store a mapping of all linknodes during the initial node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
660 # pass rather than use lookup() on the output side.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
661 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
662 filtered = []
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
663 adjustedparents = {}
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
664 linknodes = {}
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
665
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
666 for node in nodes:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
667 rev = store.rev(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
668 linknode = lookup(node)
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
669 linkrev = cl.rev(linknode)
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
670 clrevtolocalrev[linkrev] = rev
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
671
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
672 # If linknode is in fullclnodes, it means the corresponding
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
673 # changeset was a full changeset and is being sent unaltered.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
674 if linknode in fullclnodes:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
675 linknodes[node] = linknode
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
676
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
677 # If the corresponding changeset wasn't in the set computed
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
678 # as relevant to us, it should be dropped outright.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
679 elif linkrev not in precomputedellipsis:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
680 continue
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
681
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
682 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
683 # We could probably do this later and avoid the dict
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
684 # holding state. But it likely doesn't matter.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
685 p1node, p2node, linknode = _resolvenarrowrevisioninfo(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
686 cl, store, ischangelog, rev, linkrev, linknode,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
687 clrevtolocalrev, fullclnodes, precomputedellipsis)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
688
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
689 adjustedparents[node] = (p1node, p2node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
690 linknodes[node] = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
691
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
692 filtered.append(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
693
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
694 nodes = filtered
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
695
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
696 # We expect the first pass to be fast, so we only engage the progress
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
697 # meter for constructing the revision deltas.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
698 progress = None
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
699 if topic is not None:
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
700 progress = repo.ui.makeprogress(topic, unit=_('chunks'),
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
701 total=len(nodes))
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
702
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
703 configtarget = repo.ui.config('devel', 'bundle.delta')
40498
808b762679cd changegroup: add a option to create bundle with full snapshot only
Boris Feld <boris.feld@octobus.net>
parents: 40497
diff changeset
704 if configtarget not in ('', 'p1', 'full'):
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
705 msg = _("""config "devel.bundle.delta" as unknown value: %s""")
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
706 repo.ui.warn(msg % configtarget)
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
707
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
708 deltamode = repository.CG_DELTAMODE_STD
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
709 if forcedeltaparentprev:
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
710 deltamode = repository.CG_DELTAMODE_PREV
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
711 elif configtarget == 'p1':
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
712 deltamode = repository.CG_DELTAMODE_P1
40498
808b762679cd changegroup: add a option to create bundle with full snapshot only
Boris Feld <boris.feld@octobus.net>
parents: 40497
diff changeset
713 elif configtarget == 'full':
808b762679cd changegroup: add a option to create bundle with full snapshot only
Boris Feld <boris.feld@octobus.net>
parents: 40497
diff changeset
714 deltamode = repository.CG_DELTAMODE_FULL
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
715
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
716 revisions = store.emitrevisions(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
717 nodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
718 nodesorder=nodesorder,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
719 revisiondata=True,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
720 assumehaveparentrevisions=not ellipses,
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
721 deltamode=deltamode)
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
722
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
723 for i, revision in enumerate(revisions):
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
724 if progress:
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
725 progress.update(i + 1)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
726
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
727 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
728 linknode = linknodes[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
729
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
730 if revision.node in adjustedparents:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
731 p1node, p2node = adjustedparents[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
732 revision.p1node = p1node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
733 revision.p2node = p2node
40048
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39871
diff changeset
734 revision.flags |= repository.REVISION_FLAG_ELLIPSIS
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
735
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
736 else:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
737 linknode = lookup(revision.node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
738
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
739 revision.linknode = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
740 yield revision
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
741
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
742 if progress:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
743 progress.complete()
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
744
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
745 class cgpacker(object):
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
746 def __init__(self, repo, oldmatcher, matcher, version,
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
747 builddeltaheader, manifestsend,
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
748 forcedeltaparentprev=False,
38964
8a13855c8dbe changegroup: always use the treemanifest-enabled version of _packmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38949
diff changeset
749 bundlecaps=None, ellipses=False,
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
750 shallow=False, ellipsisroots=None, fullnodes=None):
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
751 """Given a source repo, construct a bundler.
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
752
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
753 oldmatcher is a matcher that matches on files the client already has.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
754 These will not be included in the changegroup.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
755
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
756 matcher is a matcher that matches on files to include in the
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
757 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
758
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
759 forcedeltaparentprev indicates whether delta parents must be against
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
760 the previous revision in a delta group. This should only be used for
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
761 compatibility with changegroup version 1.
38940
23ae0c07a3e1 changegroup: control delta parent behavior via constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38939
diff changeset
762
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
763 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
764 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
765
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
766 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
767
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
768 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
769
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
770 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
771 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
772 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
773 capabilities to customize the changegroup packer.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
774
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
775 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
776 need to pack file contents not introduced by the changes being packed.
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
777
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
778 fullnodes is the set of changelog nodes which should not be ellipsis
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
779 nodes. We store this rather than the set of nodes that should be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
780 ellipsis because for very large histories we expect this to be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
781 significantly smaller.
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
782 """
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
783 assert oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
784 assert matcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
785 self._oldmatcher = oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
786 self._matcher = matcher
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
787
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
788 self.version = version
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
789 self._forcedeltaparentprev = forcedeltaparentprev
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
790 self._builddeltaheader = builddeltaheader
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
791 self._manifestsend = manifestsend
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
792 self._ellipses = ellipses
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
793
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
794 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
795 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
796 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
797 self._bundlecaps = bundlecaps
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
798 self._isshallow = shallow
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
799 self._fullclnodes = fullnodes
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
800
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
801 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
802 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
803
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
804 self._repo = repo
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
805
23748
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
806 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
807 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
808 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
809 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
810
39688
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
811 def generate(self, commonrevs, clnodes, fastpathlinkrev, source,
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
812 changelog=True):
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
813 """Yield a sequence of changegroup byte chunks.
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
814 If changelog is False, changelog data won't be added to changegroup
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
815 """
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
816
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
817 repo = self._repo
24978
f52560c64953 changegroup: drop _changelog and _manifest properties
Martin von Zweigbergk <martinvonz@google.com>
parents: 24977
diff changeset
818 cl = repo.changelog
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
819
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
820 self._verbosenote(_('uncompressed size of bundle content:\n'))
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
821 size = 0
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
822
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
823 clstate, deltas = self._generatechangelog(cl, clnodes,
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
824 generate=changelog)
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
825 for delta in deltas:
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
826 for chunk in _revisiondeltatochunks(delta,
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
827 self._builddeltaheader):
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
828 size += len(chunk)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
829 yield chunk
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
830
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
831 close = closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
832 size += len(close)
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
833 yield closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
834
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
835 self._verbosenote(_('%8.i (changelog)\n') % size)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
836
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
837 clrevorder = clstate['clrevorder']
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
838 manifests = clstate['manifests']
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
839 changedfiles = clstate['changedfiles']
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
840
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
841 # We need to make sure that the linkrev in the changegroup refers to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
842 # the first changeset that introduced the manifest or file revision.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
843 # The fastpath is usually safer than the slowpath, because the filelogs
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
844 # are walked in revlog order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
845 #
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
846 # When taking the slowpath when the manifest revlog uses generaldelta,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
847 # the manifest may be walked in the "wrong" order. Without 'clrevorder',
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
848 # we would get an incorrect linkrev (see fix in cc0ff93d0c0c).
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
849 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
850 # When taking the fastpath, we are only vulnerable to reordering
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
851 # of the changelog itself. The changelog never uses generaldelta and is
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
852 # never reordered. To handle this case, we simply take the slowpath,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
853 # which already has the 'clrevorder' logic. This was also fixed in
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
854 # cc0ff93d0c0c.
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
855
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
856 # Treemanifests don't work correctly with fastpathlinkrev
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
857 # either, because we don't discover which directory nodes to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
858 # send along with files. This could probably be fixed.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
859 fastpathlinkrev = fastpathlinkrev and (
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
860 'treemanifest' not in repo.requirements)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
861
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
862 fnodes = {} # needed file nodes
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
863
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
864 size = 0
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
865 it = self.generatemanifests(
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
866 commonrevs, clrevorder, fastpathlinkrev, manifests, fnodes, source,
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
867 clstate['clrevtomanifestrev'])
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
868
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
869 for tree, deltas in it:
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
870 if tree:
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
871 assert self.version == b'03'
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
872 chunk = _fileheader(tree)
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
873 size += len(chunk)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
874 yield chunk
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
875
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
876 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
877 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
878 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
879 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
880 yield chunk
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
881
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
882 close = closechunk()
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
883 size += len(close)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
884 yield close
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
885
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
886 self._verbosenote(_('%8.i (manifests)\n') % size)
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
887 yield self._manifestsend
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
888
39021
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
889 mfdicts = None
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
890 if self._ellipses and self._isshallow:
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
891 mfdicts = [(self._repo.manifestlog[n].read(), lr)
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
892 for (n, lr) in manifests.iteritems()]
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
893
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
894 manifests.clear()
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
895 clrevs = set(cl.rev(x) for x in clnodes)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
896
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
897 it = self.generatefiles(changedfiles, commonrevs,
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
898 source, mfdicts, fastpathlinkrev,
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
899 fnodes, clrevs)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
900
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
901 for path, deltas in it:
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
902 h = _fileheader(path)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
903 size = len(h)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
904 yield h
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
905
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
906 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
907 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
908 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
909 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
910 yield chunk
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
911
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
912 close = closechunk()
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
913 size += len(close)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
914 yield close
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
915
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
916 self._verbosenote(_('%8.i %s\n') % (size, path))
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
917
39040
eb8a0139ace3 changegroup: inline _close()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39039
diff changeset
918 yield closechunk()
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
919
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
920 if clnodes:
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
921 repo.hook('outgoing', node=hex(clnodes[0]), source=source)
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
922
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
923 def _generatechangelog(self, cl, nodes, generate=True):
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
924 """Generate data for changelog chunks.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
925
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
926 Returns a 2-tuple of a dict containing state and an iterable of
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
927 byte chunks. The state will not be fully populated until the
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
928 chunk stream has been fully consumed.
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
929
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
930 if generate is False, the state will be fully populated and no chunk
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
931 stream will be yielded
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
932 """
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
933 clrevorder = {}
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
934 manifests = {}
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
935 mfl = self._repo.manifestlog
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
936 changedfiles = set()
39036
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39035
diff changeset
937 clrevtomanifestrev = {}
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
938
41467
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
939 state = {
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
940 'clrevorder': clrevorder,
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
941 'manifests': manifests,
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
942 'changedfiles': changedfiles,
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
943 'clrevtomanifestrev': clrevtomanifestrev,
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
944 }
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
945
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
946 if not (generate or self._ellipses):
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
947 # sort the nodes in storage order
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
948 nodes = sorted(nodes, key=cl.rev)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
949 for node in nodes:
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
950 c = cl.changelogrevision(node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
951 clrevorder[node] = len(clrevorder)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
952 # record the first changeset introducing this manifest version
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
953 manifests.setdefault(c.manifest, node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
954 # Record a complete list of potentially-changed files in
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
955 # this manifest.
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
956 changedfiles.update(c.files)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
957
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
958 return state, ()
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
959
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
960 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
961 # manifest nodes.
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
962 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
963 def lookupcl(x):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
964 c = cl.changelogrevision(x)
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
965 clrevorder[x] = len(clrevorder)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
966
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
967 if self._ellipses:
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
968 # Only update manifests if x is going to be sent. Otherwise we
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
969 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
970 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
971 # have sent.
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
972 if (x in self._fullclnodes
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
973 or cl.rev(x) in self._precomputedellipsis):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
974
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
975 manifestnode = c.manifest
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
976 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
977 # version.
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
978 manifests.setdefault(manifestnode, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
979 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
980 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
981 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
982 # parents)
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
983 clrevtomanifestrev.setdefault(
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
984 cl.rev(x), mfl.rev(manifestnode))
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
985 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
986 # client requested a shallow clone.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
987 if self._isshallow:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
988 changedfiles.update(mfl[c.manifest].read().keys())
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
989 else:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
990 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
991 else:
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
992 # record the first changeset introducing this manifest version
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
993 manifests.setdefault(c.manifest, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
994 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
995 # this manifest.
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
996 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
997
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
998 return x
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
999
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1000 gen = deltagroup(
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
1001 self._repo, cl, nodes, True, lookupcl,
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
1002 self._forcedeltaparentprev,
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1003 ellipses=self._ellipses,
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
1004 topic=_('changesets'),
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1005 clrevtolocalrev={},
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1006 fullclnodes=self._fullclnodes,
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1007 precomputedellipsis=self._precomputedellipsis)
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1008
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1009 return state, gen
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1010
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1011 def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev,
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1012 manifests, fnodes, source, clrevtolocalrev):
34161
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1013 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1014
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1015 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1016 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1017 """
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1018 repo = self._repo
30308
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30282
diff changeset
1019 mfl = repo.manifestlog
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1020 tmfnodes = {'': manifests}
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1021
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1022 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1023 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1024 # Returns the linkrev node (collected in lookupcl).
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1025 def makelookupmflinknode(tree, nodes):
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1026 if fastpathlinkrev:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1027 assert not tree
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1028 return manifests.__getitem__
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1029
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1030 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1031 """Callback for looking up the linknode for manifests.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1032
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1033 Returns the linkrev node for the specified manifest.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1034
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1035 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1036
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1037 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1038 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1039 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1040 to send
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1041
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1042 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1043 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1044 treemanifests to send.
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1045 """
35051
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1046 clnode = nodes[x]
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1047 mdata = mfl.get(tree, x).readfast(shallow=True)
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1048 for p, n, fl in mdata.iterentries():
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1049 if fl == 't': # subdirectory manifest
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1050 subtree = tree + p + '/'
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1051 tmfclnodes = tmfnodes.setdefault(subtree, {})
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1052 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1053 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1054 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1055 else:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1056 f = tree + p
28240
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1057 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1058 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1059 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1060 fclnodes[n] = clnode
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1061 return clnode
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1062 return lookupmflinknode
19206
6308896b1d4a bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents: 19204
diff changeset
1063
28232
829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents: 28231
diff changeset
1064 while tmfnodes:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1065 tree, nodes = tmfnodes.popitem()
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1066
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1067 should_visit = self._matcher.visitdir(tree[:-1])
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1068 if tree and not should_visit:
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1069 continue
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1070
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
1071 store = mfl.getstorage(tree)
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1072
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1073 if not should_visit:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1074 # No nodes to send because this directory is out of
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1075 # the client's view of the repository (probably
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1076 # because of narrow clones). Do this even for the root
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1077 # directory (tree=='')
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1078 prunednodes = []
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1079 else:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1080 # Avoid sending any manifest nodes we can prove the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1081 # client already has by checking linkrevs. See the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1082 # related comment in generatefiles().
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1083 prunednodes = self._prunemanifests(store, nodes, commonrevs)
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1084
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1085 if tree and not prunednodes:
39043
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1086 continue
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1087
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1088 lookupfn = makelookupmflinknode(tree, nodes)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1089
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1090 deltas = deltagroup(
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
1091 self._repo, store, prunednodes, False, lookupfn,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1092 self._forcedeltaparentprev,
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1093 ellipses=self._ellipses,
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
1094 topic=_('manifests'),
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1095 clrevtolocalrev=clrevtolocalrev,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1096 fullclnodes=self._fullclnodes,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1097 precomputedellipsis=self._precomputedellipsis)
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1098
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1099 if not self._oldmatcher.visitdir(store.tree[:-1]):
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1100 yield tree, deltas
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1101 else:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1102 # 'deltas' is a generator and we need to consume it even if
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1103 # we are not going to send it because a side-effect is that
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1104 # it updates tmdnodes (via lookupfn)
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1105 for d in deltas:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1106 pass
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1107 if not tree:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1108 yield tree, []
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1109
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1110 def _prunemanifests(self, store, nodes, commonrevs):
41776
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1111 if not self._ellipses:
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1112 # In non-ellipses case and large repositories, it is better to
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1113 # prevent calling of store.rev and store.linkrev on a lot of
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1114 # nodes as compared to sending some extra data
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1115 return nodes.copy()
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1116 # This is split out as a separate method to allow filtering
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1117 # commonrevs in extension code.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1118 #
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1119 # TODO(augie): this shouldn't be required, instead we should
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1120 # make filtering of revisions to send delegated to the store
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1121 # layer.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1122 frev, flr = store.rev, store.linkrev
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1123 return [n for n in nodes if flr(frev(n)) not in commonrevs]
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1124
24897
5c35a6040352 changegroup: document that 'source' parameter exists for extensions
Martin von Zweigbergk <martinvonz@google.com>
parents: 24896
diff changeset
1125 # The 'source' parameter is useful for extensions
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1126 def generatefiles(self, changedfiles, commonrevs, source,
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1127 mfdicts, fastpathlinkrev, fnodes, clrevs):
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1128 changedfiles = [f for f in changedfiles
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1129 if self._matcher(f) and not self._oldmatcher(f)]
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1130
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1131 if not fastpathlinkrev:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1132 def normallinknodes(unused, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1133 return fnodes.get(fname, {})
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1134 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1135 cln = self._repo.changelog.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1136
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1137 def normallinknodes(store, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1138 flinkrev = store.linkrev
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1139 fnode = store.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1140 revs = ((r, flinkrev(r)) for r in store)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1141 return dict((fnode(r), cln(lr))
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1142 for r, lr in revs if lr in clrevs)
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1143
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1144 clrevtolocalrev = {}
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1145
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1146 if self._isshallow:
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1147 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1148 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1149 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1150 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1151 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1152
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1153 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1154 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1155 try:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1156 fnode = c.filenode(fname)
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1157 clrevtolocalrev[c.rev()] = flog.rev(fnode)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1158 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1159 pass
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1160 links = normallinknodes(flog, fname)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1161 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1162 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1163 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1164 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1165 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1166 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1167 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1168 return links
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1169 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1170 linknodes = normallinknodes
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1171
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1172 repo = self._repo
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
1173 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
38416
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38388
diff changeset
1174 total=len(changedfiles))
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1175 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1176 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1177 if not filerevlog:
37339
5859800edfc5 changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37135
diff changeset
1178 raise error.Abort(_("empty or missing file data for %s") %
5859800edfc5 changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37135
diff changeset
1179 fname)
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1180
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1181 clrevtolocalrev.clear()
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1182
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1183 linkrevnodes = linknodes(filerevlog, fname)
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1184 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1185 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1186 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1187 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1188
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1189 frev, flr = filerevlog.rev, filerevlog.linkrev
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1190 # Skip sending any filenode we know the client already
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1191 # has. This avoids over-sending files relatively
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1192 # inexpensively, so it's not a problem if we under-filter
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1193 # here.
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1194 filenodes = [n for n in linkrevnodes
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1195 if flr(frev(n)) not in commonrevs]
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1196
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1197 if not filenodes:
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1198 continue
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1199
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1200 progress.update(i + 1, item=fname)
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1201
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1202 deltas = deltagroup(
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
1203 self._repo, filerevlog, filenodes, False, lookupfilelog,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1204 self._forcedeltaparentprev,
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1205 ellipses=self._ellipses,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1206 clrevtolocalrev=clrevtolocalrev,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1207 fullclnodes=self._fullclnodes,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1208 precomputedellipsis=self._precomputedellipsis)
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1209
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1210 yield fname, deltas
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1211
38416
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38388
diff changeset
1212 progress.complete()
19200
4cfdec944edf bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents: 19199
diff changeset
1213
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1214 def _makecg1packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1215 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1216 fullnodes=None):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1217 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1218 d.node, d.p1node, d.p2node, d.linknode)
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1219
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1220 return cgpacker(repo, oldmatcher, matcher, b'01',
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1221 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1222 manifestsend=b'',
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
1223 forcedeltaparentprev=True,
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1224 bundlecaps=bundlecaps,
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1225 ellipses=ellipses,
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
1226 shallow=shallow,
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1227 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1228 fullnodes=fullnodes)
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1229
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1230 def _makecg2packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1231 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1232 fullnodes=None):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1233 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1234 d.node, d.p1node, d.p2node, d.basenode, d.linknode)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1235
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1236 return cgpacker(repo, oldmatcher, matcher, b'02',
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1237 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1238 manifestsend=b'',
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1239 bundlecaps=bundlecaps,
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1240 ellipses=ellipses,
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
1241 shallow=shallow,
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1242 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1243 fullnodes=fullnodes)
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1244
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1245 def _makecg3packer(repo, oldmatcher, matcher, bundlecaps,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1246 ellipses=False, shallow=False, ellipsisroots=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1247 fullnodes=None):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1248 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1249 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags)
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1250
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1251 return cgpacker(repo, oldmatcher, matcher, b'03',
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1252 builddeltaheader=builddeltaheader,
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
1253 manifestsend=closechunk(),
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1254 bundlecaps=bundlecaps,
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1255 ellipses=ellipses,
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
1256 shallow=shallow,
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1257 ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1258 fullnodes=fullnodes)
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1259
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1260 _packermap = {'01': (_makecg1packer, cg1unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1261 # cg2 adds support for exchanging generaldelta
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1262 '02': (_makecg2packer, cg2unpacker),
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
1263 # cg3 adds support for exchanging revlog flags and treemanifests
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1264 '03': (_makecg3packer, cg3unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1265 }
23168
a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22971
diff changeset
1266
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1267 def allsupportedversions(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1268 versions = set(_packermap.keys())
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1269 if not (repo.ui.configbool('experimental', 'changegroup3') or
30633
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30632
diff changeset
1270 repo.ui.configbool('experimental', 'treemanifest') or
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30632
diff changeset
1271 'treemanifest' in repo.requirements):
30631
438532c99b54 changegroup: simplify 'allsupportedversions' logic
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30594
diff changeset
1272 versions.discard('03')
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1273 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1274
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1275 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1276 def supportedincomingversions(repo):
30633
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30632
diff changeset
1277 return allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1278
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1279 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1280 def supportedoutgoingversions(repo):
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1281 versions = allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1282 if 'treemanifest' in repo.requirements:
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1283 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1284 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1285 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1286 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1287 # support versions 01 and 02.
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1288 versions.discard('01')
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1289 versions.discard('02')
38874
a232e6744ba3 narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents: 38846
diff changeset
1290 if repository.NARROW_REQUIREMENT in repo.requirements:
36495
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1291 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1292 # support that for stripping and unbundling to work.
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1293 versions.discard('01')
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1294 versions.discard('02')
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1295 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1296 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1297 # mark LFS entries with REVIDX_EXTSTORED.
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1298 versions.discard('01')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1299 versions.discard('02')
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1300
27752
29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config
Martin von Zweigbergk <martinvonz@google.com>
parents: 27751
diff changeset
1301 return versions
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1302
33676
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1303 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1304 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1305 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1306 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1307
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1308 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1309 # Finds the smallest version that it's safe to assume clients of the repo
27931
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1310 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1311 # support changegroup 02.
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1312 versions = supportedoutgoingversions(repo)
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1313 if 'generaldelta' in repo.requirements:
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1314 versions.discard('01')
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1315 assert versions
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1316 return min(versions)
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1317
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1318 def getbundler(version, repo, bundlecaps=None, oldmatcher=None,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1319 matcher=None, ellipses=False, shallow=False,
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1320 ellipsisroots=None, fullnodes=None):
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1321 assert version in supportedoutgoingversions(repo)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1322
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1323 if matcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1324 matcher = matchmod.always()
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1325 if oldmatcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1326 oldmatcher = matchmod.never()
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1327
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1328 if version == '01' and not matcher.always():
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1329 raise error.ProgrammingError('version 01 changegroups do not support '
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1330 'sparse file matchers')
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1331
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1332 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1333 raise error.Abort(
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1334 _('ellipsis nodes require at least cg3 on client and server, '
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1335 'but negotiated version %s') % version)
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1336
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1337 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1338 # filter those out.
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1339 matcher = repo.narrowmatch(matcher)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1340
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1341 fn = _packermap[version][0]
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1342 return fn(repo, oldmatcher, matcher, bundlecaps, ellipses=ellipses,
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1343 shallow=shallow, ellipsisroots=ellipsisroots,
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
1344 fullnodes=fullnodes)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1345
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1346 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1347 return _packermap[version][1](fh, alg, extras=extras)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1348
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1349 def _changegroupinfo(repo, nodes, source):
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1350 if repo.ui.verbose or source == 'bundle':
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1351 repo.ui.status(_("%d changesets found\n") % len(nodes))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1352 if repo.ui.debugflag:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1353 repo.ui.debug("list of changesets:\n")
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1354 for node in nodes:
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1355 repo.ui.debug("%s\n" % hex(node))
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1356
34111
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1357 def makechangegroup(repo, outgoing, version, source, fastpath=False,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1358 bundlecaps=None):
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1359 cgstream = makestream(repo, outgoing, version, source,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1360 fastpath=fastpath, bundlecaps=bundlecaps)
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1361 return getunbundler(version, util.chunkbuffer(cgstream), None,
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1362 {'clcount': len(outgoing.missing) })
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1363
34118
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
1364 def makestream(repo, outgoing, version, source, fastpath=False,
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1365 bundlecaps=None, matcher=None):
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1366 bundler = getbundler(version, repo, bundlecaps=bundlecaps,
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1367 matcher=matcher)
34118
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
1368
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1369 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1370 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1371 csets = outgoing.missing
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1372 heads = outgoing.missingheads
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1373 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1374 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1375 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1376 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1377 fastpathlinkrev = fastpath or (
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1378 repo.filtername is None and heads == sorted(repo.heads()))
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1379
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1380 repo.hook('preoutgoing', throw=True, source=source)
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1381 _changegroupinfo(repo, csets, source)
23177
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1382 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1383
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1384 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1385 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1386 files = 0
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1387 progress = repo.ui.makeprogress(_('files'), unit=_('files'),
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1388 total=expectedfiles)
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
1389 for chunkdata in iter(source.filelogheader, {}):
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1390 files += 1
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1391 f = chunkdata["filename"]
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1392 repo.ui.debug("adding %s revisions\n" % f)
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1393 progress.increment()
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1394 fl = repo.file(f)
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1395 o = len(fl)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 23897
diff changeset
1396 try:
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1397 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1398 if not fl.addgroup(deltas, revmap, trp):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1399 raise error.Abort(_("received file revlog group is empty"))
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25624
diff changeset
1400 except error.CensoredBaseError as e:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1401 raise error.Abort(_("received delta base is censored: %s") % e)
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1402 revisions += len(fl) - o
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1403 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1404 needs = needfiles[f]
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
1405 for new in pycompat.xrange(o, len(fl)):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1406 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1407 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1408 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1409 else:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1410 raise error.Abort(
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1411 _("received spurious file revlog entry"))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1412 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1413 del needfiles[f]
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1414 progress.complete()
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1415
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1416 for f, needs in needfiles.iteritems():
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1417 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1418 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1419 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1420 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1421 except error.LookupError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1422 raise error.Abort(
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1423 _('missing file data for %s:%s - run hg verify') %
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1424 (f, hex(n)))
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1425
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1426 return revisions, files