Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 46724:a41565bef69f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
changegroup: add v4 changegroup for revlog v2 exchange This change only adds the required infrastructure for the new changegroup format and does not do any actual exchange. This will be done in the next patches. Differential Revision: https://phab.mercurial-scm.org/D10026
author Rapha?l Gom?s <rgomes@octobus.net>
date Thu, 18 Feb 2021 17:36:52 +0100
parents f4c325bf80fc
children e8c11a2c96c0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
4 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9437
diff changeset
6 # GNU General Public License version 2 or any later version.
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3859
diff changeset
7
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
9
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
10 import os
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
11 import struct
20933
d3775db748a0 localrepo: move the addchangegroup method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20932
diff changeset
12 import weakref
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
13
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
14 from .i18n import _
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
15 from .node import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
16 hex,
38922
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38874
diff changeset
17 nullid,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
18 nullrev,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
19 short,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
20 )
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
21 from .pycompat import open
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
22
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
23 from . import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
24 error,
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
25 match as matchmod,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
26 mdiff,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
27 phases,
30945
82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30743
diff changeset
28 pycompat,
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
29 requirements,
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
30 scmutil,
42823
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
31 util,
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
32 )
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
34 from .interfaces import repository
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
35 from .revlogutils import sidedata as sidedatamod
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
36
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
37 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct(b"20s20s20s20s")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
38 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct(b"20s20s20s20s20s")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
39 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(b">20s20s20s20s20sH")
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
40
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
41 LFS_REQUIREMENT = b'lfs'
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
42
35754
fb0be099063f util: move 'readexactly' in the util module
Boris Feld <boris.feld@octobus.net>
parents: 35051
diff changeset
43 readexactly = util.readexactly
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
44
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
45
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
46 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
47 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
48 d = readexactly(stream, 4)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
49 l = struct.unpack(b">l", d)[0]
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
50 if l <= 4:
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
51 if l:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
52 raise error.Abort(_(b"invalid chunk length %d") % l)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
53 return b""
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
54 return readexactly(stream, l - 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
55
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
56
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
57 def chunkheader(length):
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
58 """return a changegroup chunk header (string)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
59 return struct.pack(b">l", length + 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
60
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
61
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
62 def closechunk():
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
63 """return a changegroup chunk header (string) for a zero-length chunk"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
64 return struct.pack(b">l", 0)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
65
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
66
39019
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
67 def _fileheader(path):
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
68 """Obtain a changegroup chunk header for a named path."""
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
69 return chunkheader(len(path)) + path
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
70
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
71
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
72 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
73 """Write chunks to a file and return its filename.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
74
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
75 The stream is assumed to be a bundle file.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
76 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
77 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
78 """
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
79 fh = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
80 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
81 try:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
82 if filename:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
83 if vfs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
84 fh = vfs.open(filename, b"wb")
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
85 else:
30212
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
86 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
87 # small (4k is common on Linux).
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
88 fh = open(filename, b"wb", 131072)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
89 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
90 fd, filename = pycompat.mkstemp(prefix=b"hg-bundle-", suffix=b".hg")
43551
313e3a279828 cleanup: remove pointless r-prefixes on double-quoted strings
Augie Fackler <augie@google.com>
parents: 43538
diff changeset
91 fh = os.fdopen(fd, "wb")
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
92 cleanup = filename
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
93 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
94 fh.write(c)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
95 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
96 return filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
97 finally:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
98 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
99 fh.close()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
100 if cleanup is not None:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
101 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
102 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
103 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
104 os.unlink(cleanup)
3660
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
105
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
106
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
107 class cg1unpacker(object):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
108 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
109
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
110 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
111 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
112 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
113
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
114 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
115 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
116 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
117 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
118
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
119 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
120 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
121
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
122 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
123 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
124 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
125
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
126 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
127 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
128 version = b'01'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
129 _grouplistcount = 1 # One list of files after the manifests
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
130
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
131 def __init__(self, fh, alg, extras=None):
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
132 if alg is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
133 alg = b'UN'
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
134 if alg not in util.compengines.supportedbundletypes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
135 raise error.Abort(_(b'unknown stream compression type: %s') % alg)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
136 if alg == b'BZ':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
137 alg = b'_truncatedBZ'
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
138
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
139 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
140 self._stream = compengine.decompressorreader(fh)
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
141 self._type = alg
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
142 self.extras = extras or {}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
143 self.callback = None
26706
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
144
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
145 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
146 # be used by bundlerepo, but it's a little hard to tell.
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
147 def compressed(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
148 return self._type is not None and self._type != b'UN'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
149
12043
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
150 def read(self, l):
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
151 return self._stream.read(l)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
152
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
153 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
154 return self._stream.seek(pos)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
155
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
156 def tell(self):
12332
680fe77ab5b8 bundlerepo: use bundle objects everywhere
Matt Mackall <mpm@selenic.com>
parents: 12330
diff changeset
157 return self._stream.tell()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
158
12347
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
159 def close(self):
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
160 return self._stream.close()
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
161
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
162 def _chunklength(self):
13459
acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34
Jim Hague <jim.hague@acm.org>
parents: 13458
diff changeset
163 d = readexactly(self._stream, 4)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
164 l = struct.unpack(b">l", d)[0]
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
165 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
166 if l:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
167 raise error.Abort(_(b"invalid chunk length %d") % l)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
168 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
169 if self.callback:
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
170 self.callback()
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
171 return l - 4
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
172
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
173 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
174 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
175 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
176
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
177 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
178 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
179 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
180
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
181 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
182 """return the header of the filelogs chunk, v10 only has the filename"""
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
183 l = self._chunklength()
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
184 if not l:
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
185 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
186 fname = readexactly(self._stream, l)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
187 return {b'filename': fname}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
188
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
189 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
190 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
191 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
192 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
193 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
194 deltabase = prevnode
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
195 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
196 return node, p1, p2, deltabase, cs, flags
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
197
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
198 def deltachunk(self, prevnode):
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
199 l = self._chunklength()
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
200 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
201 return {}
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
202 headerdata = readexactly(self._stream, self.deltaheadersize)
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
203 header = self.deltaheader.unpack(headerdata)
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
204 delta = readexactly(self._stream, l - self.deltaheadersize)
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
205 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
206 # cg4 forward-compat
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
207 sidedata = {}
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
208 return (node, p1, p2, cs, deltabase, delta, flags, sidedata)
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
209
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
210 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
211 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
212
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
213 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
214 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
215 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
216 """
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
217 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
218 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
219 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
220 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
221 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
222 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
223 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
224 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
225 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
226 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
227 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
228
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
229 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
230 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
231 noentries = True
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
232 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
233 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
234 if not chunk:
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
235 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
236 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
237 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
238 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
239 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
240 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
241 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
242 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
243 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
244 parts += 1
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
245 break
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
246 noentries = False
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
247 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
248 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
249 while pos < len(chunk):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
250 next = pos + 2 ** 20
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
251 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
252 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
253 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
254
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
255 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
256 self.callback = prog.increment
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
257 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
258 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
259 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
260 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
261 self.manifestheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
262 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
263 repo.manifestlog.getstorage(b'').addgroup(deltas, revmap, trp)
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
264 prog.complete()
28360
11287888ce4b changegroup: exclude submanifests from manifest progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 28281
diff changeset
265 self.callback = None
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
266
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
267 def apply(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
268 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
269 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
270 tr,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
271 srctype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
272 url,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
273 targetphase=phases.draft,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
274 expectedtotal=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
275 ):
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
276 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
277 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
278 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
279
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
280 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
281 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
282 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
283 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
284 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
285 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
286 repo = repo.unfiltered()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
287
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
288 def csmap(x):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
289 repo.ui.debug(b"add changeset %s\n" % short(x))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
290 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
291
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
292 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
293 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
294
26880
fa7f8b686633 changegroup: fix the scope of a try finally
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26859
diff changeset
295 try:
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
296 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
297 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
298 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
299 # in this function.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
300 srctype = tr.hookargs.setdefault(b'source', srctype)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
301 tr.hookargs.setdefault(b'url', url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
302 repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
303 b'prechangegroup', throw=True, **pycompat.strkwargs(tr.hookargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
304 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
305
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
306 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
307 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
308 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
309 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
310 oldheads = set(cl.heads())
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
311
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
312 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
313 # pull off the changeset group
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
314 repo.ui.status(_(b"adding changesets\n"))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
315 clstart = len(cl)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
316 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
317 _(b'changesets'), unit=_(b'chunks'), total=expectedtotal
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
318 )
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
319 self.callback = progress.increment
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
320
43537
ea0b44255a31 changegroup: avoid shadowing a set with an int
Augie Fackler <augie@google.com>
parents: 43132
diff changeset
321 efilesset = set()
46562
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
322 duprevs = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
323
46561
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46445
diff changeset
324 def ondupchangelog(cl, rev):
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46445
diff changeset
325 if rev < clstart:
46562
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
326 duprevs.append(rev)
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
327
46561
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46445
diff changeset
328 def onchangelog(cl, rev):
46443
0903d6b9b1df repository: introduce register_changeset callback
Joerg Sonnenberger <joerg@bec.de>
parents: 45813
diff changeset
329 ctx = cl.changelogrevision(rev)
0903d6b9b1df repository: introduce register_changeset callback
Joerg Sonnenberger <joerg@bec.de>
parents: 45813
diff changeset
330 efilesset.update(ctx.files)
0903d6b9b1df repository: introduce register_changeset callback
Joerg Sonnenberger <joerg@bec.de>
parents: 45813
diff changeset
331 repo.register_changeset(rev, ctx)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
332
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
333 self.changelogheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
334 deltas = self.deltaiter()
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
335 if not cl.addgroup(
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
336 deltas,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
337 csmap,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
338 trp,
46445
711ba0f1057e revlog: decouple caching from addrevision callback for addgroup
Joerg Sonnenberger <joerg@bec.de>
parents: 46443
diff changeset
339 alwayscache=True,
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
340 addrevisioncb=onchangelog,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
341 duplicaterevisioncb=ondupchangelog,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
342 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
343 repo.ui.develwarn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
344 b'applied empty changelog from changegroup',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
345 config=b'warn-empty-changegroup',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
346 )
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
347 efiles = len(efilesset)
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
348 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
349 changesets = clend - clstart
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
350 progress.complete()
45729
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
351 del deltas
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
352 # TODO Python 2.7 removal
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
353 # del efilesset
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
354 efilesset = None
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
355 self.callback = None
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
356
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
357 # pull off the manifest group
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
358 repo.ui.status(_(b"adding manifests\n"))
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
359 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
360 # changesets.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
361 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
362 _(b'manifests'), unit=_(b'chunks'), total=changesets
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
363 )
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
364 self._unpackmanifests(repo, revmap, trp, progress)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
365
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
366 needfiles = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
367 if repo.ui.configbool(b'server', b'validate'):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
368 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
369 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
370 # validate incoming csets have their manifests
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
371 for cset in pycompat.xrange(clstart, clend):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
372 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
373 mfest = ml[mfnode].readdelta()
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
374 # store file nodes we must see
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
375 for f, n in pycompat.iteritems(mfest):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
376 needfiles.setdefault(f, set()).add(n)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
377
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
378 # process the files
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
379 repo.ui.status(_(b"adding file changes\n"))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
380 newrevs, newfiles = _addchangegroupfiles(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
381 repo, self, revmap, trp, efiles, needfiles
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
382 )
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
383
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
384 # making sure the value exists
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
385 tr.changes.setdefault(b'changegroup-count-changesets', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
386 tr.changes.setdefault(b'changegroup-count-revisions', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
387 tr.changes.setdefault(b'changegroup-count-files', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
388 tr.changes.setdefault(b'changegroup-count-heads', 0)
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
389
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
390 # some code use bundle operation for internal purpose. They usually
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
391 # set `ui.quiet` to do this outside of user sight. Size the report
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
392 # of such operation now happens at the end of the transaction, that
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
393 # ui.quiet has not direct effect on the output.
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
394 #
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
395 # To preserve this intend use an inelegant hack, we fail to report
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
396 # the change if `quiet` is set. We should probably move to
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
397 # something better, but this is a good first step to allow the "end
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
398 # of transaction report" to pass tests.
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
399 if not repo.ui.quiet:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
400 tr.changes[b'changegroup-count-changesets'] += changesets
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
401 tr.changes[b'changegroup-count-revisions'] += newrevs
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
402 tr.changes[b'changegroup-count-files'] += newfiles
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
403
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
404 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
405 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
406 heads = cl.heads()
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
407 deltaheads += len(heads) - len(oldheads)
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
408 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
409 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
410 deltaheads -= 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
411
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
412 # see previous comment about checking ui.quiet
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
413 if not repo.ui.quiet:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
414 tr.changes[b'changegroup-count-heads'] += deltaheads
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
415 repo.invalidatevolatilesets()
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
416
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
417 if changesets > 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
418 if b'node' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
419 tr.hookargs[b'node'] = hex(cl.node(clstart))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
420 tr.hookargs[b'node_last'] = hex(cl.node(clend - 1))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
421 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
422 else:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
423 hookargs = dict(tr.hookargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
424 hookargs[b'node'] = hex(cl.node(clstart))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
425 hookargs[b'node_last'] = hex(cl.node(clend - 1))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
426 repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
427 b'pretxnchangegroup',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
428 throw=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
429 **pycompat.strkwargs(hookargs)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
430 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
431
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
432 added = pycompat.xrange(clstart, clend)
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
433 phaseall = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
434 if srctype in (b'push', b'serve'):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
435 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
436 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
437 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
438 #
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
439 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
440 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
441 if repo.publishing():
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
442 targetphase = phaseall = phases.public
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
443 else:
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
444 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
445
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
446 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
447 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
448 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
449 # ignored.
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
450 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
451 if added:
45813
5d65e04b6a80 phases: convert registernew users to use revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45812
diff changeset
452 phases.registernew(repo, tr, targetphase, added)
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
453 if phaseall is not None:
46562
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
454 if duprevs:
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
455 duprevs.extend(added)
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
456 else:
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
457 duprevs = added
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
458 phases.advanceboundary(repo, tr, phaseall, [], revs=duprevs)
fa7ae7aa0efd changegroup: don't convert revisions to node for duplicate handling
Joerg Sonnenberger <joerg@bec.de>
parents: 46561
diff changeset
459 duprevs = []
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
460
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
461 if changesets > 0:
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
462
43798
888bd39ed555 lock: pass "success" boolean to _afterlock callbacks
Kyle Lippincott <spectral@google.com>
parents: 43551
diff changeset
463 def runhooks(unused_success):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
464 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
465 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
466 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
467 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
468 return
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
469
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
470 repo.hook(b"changegroup", **pycompat.strkwargs(hookargs))
27867
7ced54ebf972 with: use context manager for transaction in changegroup apply
Bryan O'Sullivan <bryano@fb.com>
parents: 27754
diff changeset
471
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
472 for rev in added:
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
473 args = hookargs.copy()
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
474 args[b'node'] = hex(cl.node(rev))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
475 del args[b'node_last']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
476 repo.hook(b"incoming", **pycompat.strkwargs(args))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
477
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
478 newheads = [h for h in repo.heads() if h not in oldheads]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
479 repo.ui.log(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
480 b"incoming",
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
481 b"%d incoming changes - new heads: %s\n",
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
482 len(added),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
483 b', '.join([hex(c[:6]) for c in newheads]),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
484 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
485
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
486 tr.addpostclose(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
487 b'changegroup-runhooks-%020i' % clstart,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
488 lambda tr: repo._afterlock(runhooks),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
489 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
490 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
491 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
492 # never return 0 here:
32888
b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32887
diff changeset
493 if deltaheads < 0:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
494 ret = deltaheads - 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
495 else:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
496 ret = deltaheads + 1
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33456
diff changeset
497 return ret
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
498
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
499 def deltaiter(self):
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
500 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
501 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
502
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
503 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
504 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
505 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
506 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34301
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
507 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
508 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
509 chain = chunkdata[0]
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
510
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
511
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
512 class cg2unpacker(cg1unpacker):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
513 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
514
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
515 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
516 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
517 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
518 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
519
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
520 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
521 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
522 version = b'02'
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
523
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
524 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
525 node, p1, p2, deltabase, cs = headertuple
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
526 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
527 return node, p1, p2, deltabase, cs, flags
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
528
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
529
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
530 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
531 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
532
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
533 cg3 streams add support for exchanging treemanifests and revlog
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
534 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
535 separating manifests and files.
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
536 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
537
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
538 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
539 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
540 version = b'03'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
541 _grouplistcount = 2 # One list of manifests and one list of files
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
542
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
543 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
544 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
545 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
546
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
547 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
548 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
549 for chunkdata in iter(self.filelogheader, {}):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
550 # If we get here, there are directory manifests in the changegroup
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
551 d = chunkdata[b"filename"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
552 repo.ui.debug(b"adding %s revisions\n" % d)
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
553 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
554 if not repo.manifestlog.getstorage(d).addgroup(deltas, revmap, trp):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
555 raise error.Abort(_(b"received dir revlog group is empty"))
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
556
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
557
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
558 class cg4unpacker(cg3unpacker):
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
559 """Unpacker for cg4 streams.
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
560
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
561 cg4 streams add support for exchanging sidedata.
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
562 """
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
563
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
564 version = b'04'
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
565
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
566 def deltachunk(self, prevnode):
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
567 res = super(cg4unpacker, self).deltachunk(prevnode)
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
568 if not res:
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
569 return res
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
570
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
571 (node, p1, p2, cs, deltabase, delta, flags, _sidedata) = res
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
572
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
573 sidedata_raw = getchunk(self._stream)
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
574 sidedata = {}
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
575 if len(sidedata_raw) > 0:
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
576 sidedata = sidedatamod.deserialize_sidedata(sidedata_raw)
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
577
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
578 return node, p1, p2, cs, deltabase, delta, flags, sidedata
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
579
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
580
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
581 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
582 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
583 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
584 self._fh = fh
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
585
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
586 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
587 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
588 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
589 if len(d) < n:
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
590 d += readexactly(self._fh, n - len(d))
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
591 return d
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
592 return readexactly(self._fh, n)
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
593
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
594
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
595 def _revisiondeltatochunks(delta, headerfn):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
596 """Serialize a revisiondelta to changegroup chunks."""
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
597
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
598 # The captured revision delta may be encoded as a delta against
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
599 # a base revision or as a full revision. The changegroup format
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
600 # requires that everything on the wire be deltas. So for full
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
601 # revisions, we need to invent a header that says to rewrite
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
602 # data.
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
603
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
604 if delta.delta is not None:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
605 prefix, data = b'', delta.delta
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
606 elif delta.basenode == nullid:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
607 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
608 prefix = mdiff.trivialdiffheader(len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
609 else:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
610 data = delta.revision
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
611 prefix = mdiff.replacediffheader(delta.baserevisionsize, len(data))
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
612
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
613 meta = headerfn(delta)
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
614
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
615 yield chunkheader(len(meta) + len(prefix) + len(data))
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
616 yield meta
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
617 if prefix:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
618 yield prefix
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
619 yield data
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
620
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
621
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
622 def _sortnodesellipsis(store, nodes, cl, lookup):
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
623 """Sort nodes for changegroup generation."""
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
624 # Ellipses serving mode.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
625 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
626 # In a perfect world, we'd generate better ellipsis-ified graphs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
627 # for non-changelog revlogs. In practice, we haven't started doing
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
628 # that yet, so the resulting DAGs for the manifestlog and filelogs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
629 # are actually full of bogus parentage on all the ellipsis
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
630 # nodes. This has the side effect that, while the contents are
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
631 # correct, the individual DAGs might be completely out of whack in
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
632 # a case like 882681bc3166 and its ancestors (back about 10
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
633 # revisions or so) in the main hg repo.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
634 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
635 # The one invariant we *know* holds is that the new (potentially
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
636 # bogus) DAG shape will be valid if we order the nodes in the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
637 # order that they're introduced in dramatis personae by the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
638 # changelog, so what we do is we sort the non-changelog histories
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
639 # by the order in which they are used by the changelog.
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
640 key = lambda n: cl.rev(lookup(n))
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
641 return sorted(nodes, key=key)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
642
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
643
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
644 def _resolvenarrowrevisioninfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
645 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
646 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
647 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
648 rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
649 linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
650 linknode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
651 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
652 fullclnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
653 precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
654 ):
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
655 linkparents = precomputedellipsis[linkrev]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
656
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
657 def local(clrev):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
658 """Turn a changelog revnum into a local revnum.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
659
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
660 The ellipsis dag is stored as revnums on the changelog,
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
661 but when we're producing ellipsis entries for
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
662 non-changelog revlogs, we need to turn those numbers into
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
663 something local. This does that for us, and during the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
664 changelog sending phase will also expand the stored
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
665 mappings as needed.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
666 """
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
667 if clrev == nullrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
668 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
669
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
670 if ischangelog:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
671 return clrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
672
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
673 # Walk the ellipsis-ized changelog breadth-first looking for a
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
674 # change that has been linked from the current revlog.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
675 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
676 # For a flat manifest revlog only a single step should be necessary
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
677 # as all relevant changelog entries are relevant to the flat
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
678 # manifest.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
679 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
680 # For a filelog or tree manifest dirlog however not every changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
681 # entry will have been relevant, so we need to skip some changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
682 # nodes even after ellipsis-izing.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
683 walk = [clrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
684 while walk:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
685 p = walk[0]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
686 walk = walk[1:]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
687 if p in clrevtolocalrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
688 return clrevtolocalrev[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
689 elif p in fullclnodes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
690 walk.extend([pp for pp in cl.parentrevs(p) if pp != nullrev])
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
691 elif p in precomputedellipsis:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
692 walk.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
693 [pp for pp in precomputedellipsis[p] if pp != nullrev]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
694 )
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
695 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
696 # In this case, we've got an ellipsis with parents
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
697 # outside the current bundle (likely an
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
698 # incremental pull). We "know" that we can use the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
699 # value of this same revlog at whatever revision
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
700 # is pointed to by linknode. "Know" is in scare
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
701 # quotes because I haven't done enough examination
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
702 # of edge cases to convince myself this is really
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
703 # a fact - it works for all the (admittedly
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
704 # thorough) cases in our testsuite, but I would be
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
705 # somewhat unsurprised to find a case in the wild
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
706 # where this breaks down a bit. That said, I don't
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
707 # know if it would hurt anything.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
708 for i in pycompat.xrange(rev, 0, -1):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
709 if store.linkrev(i) == clrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
710 return i
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
711 # We failed to resolve a parent for this node, so
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
712 # we crash the changegroup construction.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
713 raise error.Abort(
46565
c3c7a86e9c24 tests: fix differing output between py2 and py3
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46562
diff changeset
714 b"unable to resolve parent while packing '%s' %r"
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
715 b' for changeset %r' % (store.indexfile, rev, clrev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
716 )
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
717
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
718 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
719
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
720 if not linkparents or (store.parentrevs(rev) == (nullrev, nullrev)):
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
721 p1, p2 = nullrev, nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
722 elif len(linkparents) == 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
723 (p1,) = sorted(local(p) for p in linkparents)
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
724 p2 = nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
725 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
726 p1, p2 = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
727
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
728 p1node, p2node = store.node(p1), store.node(p2)
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
729
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
730 return p1node, p2node, linknode
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
731
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
732
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
733 def deltagroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
734 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
735 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
736 nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
737 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
738 lookup,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
739 forcedeltaparentprev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
740 topic=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
741 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
742 clrevtolocalrev=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
743 fullclnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
744 precomputedellipsis=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
745 ):
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
746 """Calculate deltas for a set of revisions.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
747
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
748 Is a generator of ``revisiondelta`` instances.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
749
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
750 If topic is not None, progress detail will be generated using this
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
751 topic name (e.g. changesets, manifests, etc).
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
752 """
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
753 if not nodes:
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
754 return
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
755
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
756 cl = repo.changelog
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
757
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
758 if ischangelog:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
759 # `hg log` shows changesets in storage order. To preserve order
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
760 # across clones, send out changesets in storage order.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
761 nodesorder = b'storage'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
762 elif ellipses:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
763 nodes = _sortnodesellipsis(store, nodes, cl, lookup)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
764 nodesorder = b'nodes'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
765 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
766 nodesorder = None
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
767
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
768 # Perform ellipses filtering and revision massaging. We do this before
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
769 # emitrevisions() because a) filtering out revisions creates less work
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
770 # for emitrevisions() b) dropping revisions would break emitrevisions()'s
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
771 # assumptions about delta choices and we would possibly send a delta
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
772 # referencing a missing base revision.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
773 #
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
774 # Also, calling lookup() has side-effects with regards to populating
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
775 # data structures. If we don't call lookup() for each node or if we call
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
776 # lookup() after the first pass through each node, things can break -
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
777 # possibly intermittently depending on the python hash seed! For that
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
778 # reason, we store a mapping of all linknodes during the initial node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
779 # pass rather than use lookup() on the output side.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
780 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
781 filtered = []
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
782 adjustedparents = {}
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
783 linknodes = {}
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
784
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
785 for node in nodes:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
786 rev = store.rev(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
787 linknode = lookup(node)
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
788 linkrev = cl.rev(linknode)
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
789 clrevtolocalrev[linkrev] = rev
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
790
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
791 # If linknode is in fullclnodes, it means the corresponding
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
792 # changeset was a full changeset and is being sent unaltered.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
793 if linknode in fullclnodes:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
794 linknodes[node] = linknode
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
795
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
796 # If the corresponding changeset wasn't in the set computed
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
797 # as relevant to us, it should be dropped outright.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
798 elif linkrev not in precomputedellipsis:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
799 continue
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
800
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
801 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
802 # We could probably do this later and avoid the dict
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
803 # holding state. But it likely doesn't matter.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
804 p1node, p2node, linknode = _resolvenarrowrevisioninfo(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
805 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
806 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
807 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
808 rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
809 linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
810 linknode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
811 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
812 fullclnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
813 precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
814 )
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
815
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
816 adjustedparents[node] = (p1node, p2node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
817 linknodes[node] = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
818
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
819 filtered.append(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
820
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
821 nodes = filtered
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
822
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
823 # We expect the first pass to be fast, so we only engage the progress
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
824 # meter for constructing the revision deltas.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
825 progress = None
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
826 if topic is not None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
827 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
828 topic, unit=_(b'chunks'), total=len(nodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
829 )
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
830
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
831 configtarget = repo.ui.config(b'devel', b'bundle.delta')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
832 if configtarget not in (b'', b'p1', b'full'):
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
833 msg = _("""config "devel.bundle.delta" as unknown value: %s""")
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
834 repo.ui.warn(msg % configtarget)
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
835
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
836 deltamode = repository.CG_DELTAMODE_STD
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
837 if forcedeltaparentprev:
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
838 deltamode = repository.CG_DELTAMODE_PREV
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
839 elif configtarget == b'p1':
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
840 deltamode = repository.CG_DELTAMODE_P1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
841 elif configtarget == b'full':
40498
808b762679cd changegroup: add a option to create bundle with full snapshot only
Boris Feld <boris.feld@octobus.net>
parents: 40497
diff changeset
842 deltamode = repository.CG_DELTAMODE_FULL
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
843
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
844 revisions = store.emitrevisions(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
845 nodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
846 nodesorder=nodesorder,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
847 revisiondata=True,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
848 assumehaveparentrevisions=not ellipses,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
849 deltamode=deltamode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
850 )
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
851
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
852 for i, revision in enumerate(revisions):
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
853 if progress:
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
854 progress.update(i + 1)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
855
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
856 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
857 linknode = linknodes[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
858
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
859 if revision.node in adjustedparents:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
860 p1node, p2node = adjustedparents[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
861 revision.p1node = p1node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
862 revision.p2node = p2node
40048
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39871
diff changeset
863 revision.flags |= repository.REVISION_FLAG_ELLIPSIS
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
864
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
865 else:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
866 linknode = lookup(revision.node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
867
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
868 revision.linknode = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
869 yield revision
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
870
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
871 if progress:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
872 progress.complete()
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
873
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
874
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
875 class cgpacker(object):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
876 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
877 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
878 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
879 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
880 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
881 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
882 builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
883 manifestsend,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
884 forcedeltaparentprev=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
885 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
886 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
887 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
888 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
889 fullnodes=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
890 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
891 ):
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
892 """Given a source repo, construct a bundler.
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
893
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
894 oldmatcher is a matcher that matches on files the client already has.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
895 These will not be included in the changegroup.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
896
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
897 matcher is a matcher that matches on files to include in the
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
898 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
899
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
900 forcedeltaparentprev indicates whether delta parents must be against
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
901 the previous revision in a delta group. This should only be used for
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
902 compatibility with changegroup version 1.
38940
23ae0c07a3e1 changegroup: control delta parent behavior via constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38939
diff changeset
903
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
904 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
905 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
906
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
907 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
908
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
909 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
910
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
911 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
912 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
913 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
914 capabilities to customize the changegroup packer.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
915
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
916 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
917 need to pack file contents not introduced by the changes being packed.
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
918
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
919 fullnodes is the set of changelog nodes which should not be ellipsis
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
920 nodes. We store this rather than the set of nodes that should be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
921 ellipsis because for very large histories we expect this to be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
922 significantly smaller.
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
923
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
924 remote_sidedata is the set of sidedata categories wanted by the remote.
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
925 """
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
926 assert oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
927 assert matcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
928 self._oldmatcher = oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
929 self._matcher = matcher
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
930
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
931 self.version = version
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
932 self._forcedeltaparentprev = forcedeltaparentprev
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
933 self._builddeltaheader = builddeltaheader
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
934 self._manifestsend = manifestsend
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
935 self._ellipses = ellipses
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
936
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
937 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
938 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
939 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
940 self._bundlecaps = bundlecaps
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
941 self._isshallow = shallow
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
942 self._fullclnodes = fullnodes
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
943
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
944 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
945 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
946
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
947 self._repo = repo
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
948
23748
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
949 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
950 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
951 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
952 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
953
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
954 def generate(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
955 self, commonrevs, clnodes, fastpathlinkrev, source, changelog=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
956 ):
39688
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
957 """Yield a sequence of changegroup byte chunks.
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
958 If changelog is False, changelog data won't be added to changegroup
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
959 """
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
960
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
961 repo = self._repo
24978
f52560c64953 changegroup: drop _changelog and _manifest properties
Martin von Zweigbergk <martinvonz@google.com>
parents: 24977
diff changeset
962 cl = repo.changelog
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
963
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
964 self._verbosenote(_(b'uncompressed size of bundle content:\n'))
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
965 size = 0
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
966
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
967 clstate, deltas = self._generatechangelog(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
968 cl, clnodes, generate=changelog
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
969 )
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
970 for delta in deltas:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
971 for chunk in _revisiondeltatochunks(delta, self._builddeltaheader):
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
972 size += len(chunk)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
973 yield chunk
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
974
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
975 close = closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
976 size += len(close)
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
977 yield closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
978
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
979 self._verbosenote(_(b'%8.i (changelog)\n') % size)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
980
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
981 clrevorder = clstate[b'clrevorder']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
982 manifests = clstate[b'manifests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
983 changedfiles = clstate[b'changedfiles']
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
984
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
985 # We need to make sure that the linkrev in the changegroup refers to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
986 # the first changeset that introduced the manifest or file revision.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
987 # The fastpath is usually safer than the slowpath, because the filelogs
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
988 # are walked in revlog order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
989 #
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
990 # When taking the slowpath when the manifest revlog uses generaldelta,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
991 # the manifest may be walked in the "wrong" order. Without 'clrevorder',
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
992 # we would get an incorrect linkrev (see fix in cc0ff93d0c0c).
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
993 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
994 # When taking the fastpath, we are only vulnerable to reordering
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
995 # of the changelog itself. The changelog never uses generaldelta and is
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
996 # never reordered. To handle this case, we simply take the slowpath,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
997 # which already has the 'clrevorder' logic. This was also fixed in
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
998 # cc0ff93d0c0c.
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
999
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1000 # Treemanifests don't work correctly with fastpathlinkrev
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1001 # either, because we don't discover which directory nodes to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1002 # send along with files. This could probably be fixed.
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
1003 fastpathlinkrev = fastpathlinkrev and not scmutil.istreemanifest(repo)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1004
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1005 fnodes = {} # needed file nodes
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1006
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
1007 size = 0
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1008 it = self.generatemanifests(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1009 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1010 clrevorder,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1011 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1012 manifests,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1013 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1014 source,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1015 clstate[b'clrevtomanifestrev'],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1016 )
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1017
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1018 for tree, deltas in it:
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1019 if tree:
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1020 assert self.version in (b'03', b'04')
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1021 chunk = _fileheader(tree)
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1022 size += len(chunk)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1023 yield chunk
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1024
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1025 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1026 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1027 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1028 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1029 yield chunk
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1030
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1031 close = closechunk()
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1032 size += len(close)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
1033 yield close
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1034
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1035 self._verbosenote(_(b'%8.i (manifests)\n') % size)
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
1036 yield self._manifestsend
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
1037
39021
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
1038 mfdicts = None
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
1039 if self._ellipses and self._isshallow:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1040 mfdicts = [
46663
357d2ea95ce9 changegroup: use the local variable instead of reaching through self
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46565
diff changeset
1041 (repo.manifestlog[n].read(), lr)
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
1042 for (n, lr) in pycompat.iteritems(manifests)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1043 ]
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1044
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1045 manifests.clear()
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44349
diff changeset
1046 clrevs = {cl.rev(x) for x in clnodes}
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1047
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1048 it = self.generatefiles(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1049 changedfiles,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1050 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1051 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1052 mfdicts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1053 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1054 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1055 clrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1056 )
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1057
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1058 for path, deltas in it:
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1059 h = _fileheader(path)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1060 size = len(h)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1061 yield h
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1062
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1063 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1064 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1065 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1066 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1067 yield chunk
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1068
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1069 close = closechunk()
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1070 size += len(close)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1071 yield close
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1072
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1073 self._verbosenote(_(b'%8.i %s\n') % (size, path))
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1074
39040
eb8a0139ace3 changegroup: inline _close()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39039
diff changeset
1075 yield closechunk()
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1076
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1077 if clnodes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1078 repo.hook(b'outgoing', node=hex(clnodes[0]), source=source)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1079
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1080 def _generatechangelog(self, cl, nodes, generate=True):
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1081 """Generate data for changelog chunks.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1082
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1083 Returns a 2-tuple of a dict containing state and an iterable of
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1084 byte chunks. The state will not be fully populated until the
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1085 chunk stream has been fully consumed.
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1086
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1087 if generate is False, the state will be fully populated and no chunk
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1088 stream will be yielded
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1089 """
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1090 clrevorder = {}
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1091 manifests = {}
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1092 mfl = self._repo.manifestlog
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1093 changedfiles = set()
39036
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39035
diff changeset
1094 clrevtomanifestrev = {}
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1095
41467
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1096 state = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1097 b'clrevorder': clrevorder,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1098 b'manifests': manifests,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1099 b'changedfiles': changedfiles,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1100 b'clrevtomanifestrev': clrevtomanifestrev,
41467
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1101 }
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1102
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1103 if not (generate or self._ellipses):
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1104 # sort the nodes in storage order
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1105 nodes = sorted(nodes, key=cl.rev)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1106 for node in nodes:
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1107 c = cl.changelogrevision(node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1108 clrevorder[node] = len(clrevorder)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1109 # record the first changeset introducing this manifest version
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1110 manifests.setdefault(c.manifest, node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1111 # Record a complete list of potentially-changed files in
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1112 # this manifest.
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1113 changedfiles.update(c.files)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1114
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1115 return state, ()
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1116
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1117 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1118 # manifest nodes.
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1119 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1120 def lookupcl(x):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1121 c = cl.changelogrevision(x)
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1122 clrevorder[x] = len(clrevorder)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1123
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1124 if self._ellipses:
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1125 # Only update manifests if x is going to be sent. Otherwise we
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1126 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1127 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1128 # have sent.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1129 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1130 x in self._fullclnodes
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1131 or cl.rev(x) in self._precomputedellipsis
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1132 ):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1133
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1134 manifestnode = c.manifest
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1135 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1136 # version.
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1137 manifests.setdefault(manifestnode, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1138 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1139 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1140 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1141 # parents)
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1142 clrevtomanifestrev.setdefault(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1143 cl.rev(x), mfl.rev(manifestnode)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1144 )
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1145 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1146 # client requested a shallow clone.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1147 if self._isshallow:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1148 changedfiles.update(mfl[c.manifest].read().keys())
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1149 else:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1150 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1151 else:
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1152 # record the first changeset introducing this manifest version
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1153 manifests.setdefault(c.manifest, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1154 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1155 # this manifest.
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1156 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1157
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1158 return x
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1159
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1160 gen = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1161 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1162 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1163 nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1164 True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1165 lookupcl,
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
1166 self._forcedeltaparentprev,
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1167 ellipses=self._ellipses,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1168 topic=_(b'changesets'),
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1169 clrevtolocalrev={},
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1170 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1171 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1172 )
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1173
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1174 return state, gen
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1175
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1176 def generatemanifests(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1177 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1178 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1179 clrevorder,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1180 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1181 manifests,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1182 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1183 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1184 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1185 ):
34161
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1186 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1187
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1188 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1189 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1190 """
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1191 repo = self._repo
30308
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30282
diff changeset
1192 mfl = repo.manifestlog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1193 tmfnodes = {b'': manifests}
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1194
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1195 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1196 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1197 # Returns the linkrev node (collected in lookupcl).
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1198 def makelookupmflinknode(tree, nodes):
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1199 if fastpathlinkrev:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1200 assert not tree
43538
5b5e62c24b2e changegroup: suppress pytype error that's wrong
Augie Fackler <augie@google.com>
parents: 43537
diff changeset
1201 return (
44349
a0ec05d93c8e cleanup: re-run black on the codebase
Augie Fackler <augie@google.com>
parents: 43976
diff changeset
1202 manifests.__getitem__
a0ec05d93c8e cleanup: re-run black on the codebase
Augie Fackler <augie@google.com>
parents: 43976
diff changeset
1203 ) # pytype: disable=unsupported-operands
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1204
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1205 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1206 """Callback for looking up the linknode for manifests.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1207
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1208 Returns the linkrev node for the specified manifest.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1209
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1210 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1211
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1212 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1213 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1214 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1215 to send
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1216
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1217 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1218 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1219 treemanifests to send.
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1220 """
35051
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1221 clnode = nodes[x]
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1222 mdata = mfl.get(tree, x).readfast(shallow=True)
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1223 for p, n, fl in mdata.iterentries():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1224 if fl == b't': # subdirectory manifest
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1225 subtree = tree + p + b'/'
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1226 tmfclnodes = tmfnodes.setdefault(subtree, {})
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1227 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1228 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1229 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1230 else:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1231 f = tree + p
28240
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1232 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1233 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1234 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1235 fclnodes[n] = clnode
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1236 return clnode
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1237
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1238 return lookupmflinknode
19206
6308896b1d4a bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents: 19204
diff changeset
1239
28232
829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents: 28231
diff changeset
1240 while tmfnodes:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1241 tree, nodes = tmfnodes.popitem()
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1242
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1243 should_visit = self._matcher.visitdir(tree[:-1])
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1244 if tree and not should_visit:
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1245 continue
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1246
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
1247 store = mfl.getstorage(tree)
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1248
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1249 if not should_visit:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1250 # No nodes to send because this directory is out of
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1251 # the client's view of the repository (probably
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1252 # because of narrow clones). Do this even for the root
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1253 # directory (tree=='')
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1254 prunednodes = []
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1255 else:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1256 # Avoid sending any manifest nodes we can prove the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1257 # client already has by checking linkrevs. See the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1258 # related comment in generatefiles().
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1259 prunednodes = self._prunemanifests(store, nodes, commonrevs)
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1260
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1261 if tree and not prunednodes:
39043
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1262 continue
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1263
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1264 lookupfn = makelookupmflinknode(tree, nodes)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1265
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1266 deltas = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1267 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1268 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1269 prunednodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1270 False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1271 lookupfn,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1272 self._forcedeltaparentprev,
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1273 ellipses=self._ellipses,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1274 topic=_(b'manifests'),
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1275 clrevtolocalrev=clrevtolocalrev,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1276 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1277 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1278 )
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1279
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1280 if not self._oldmatcher.visitdir(store.tree[:-1]):
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1281 yield tree, deltas
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1282 else:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1283 # 'deltas' is a generator and we need to consume it even if
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1284 # we are not going to send it because a side-effect is that
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1285 # it updates tmdnodes (via lookupfn)
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1286 for d in deltas:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1287 pass
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1288 if not tree:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1289 yield tree, []
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1290
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1291 def _prunemanifests(self, store, nodes, commonrevs):
41776
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1292 if not self._ellipses:
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1293 # In non-ellipses case and large repositories, it is better to
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1294 # prevent calling of store.rev and store.linkrev on a lot of
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1295 # nodes as compared to sending some extra data
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1296 return nodes.copy()
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1297 # This is split out as a separate method to allow filtering
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1298 # commonrevs in extension code.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1299 #
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1300 # TODO(augie): this shouldn't be required, instead we should
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1301 # make filtering of revisions to send delegated to the store
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1302 # layer.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1303 frev, flr = store.rev, store.linkrev
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1304 return [n for n in nodes if flr(frev(n)) not in commonrevs]
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1305
24897
5c35a6040352 changegroup: document that 'source' parameter exists for extensions
Martin von Zweigbergk <martinvonz@google.com>
parents: 24896
diff changeset
1306 # The 'source' parameter is useful for extensions
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1307 def generatefiles(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1308 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1309 changedfiles,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1310 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1311 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1312 mfdicts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1313 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1314 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1315 clrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1316 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1317 changedfiles = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1318 f
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1319 for f in changedfiles
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1320 if self._matcher(f) and not self._oldmatcher(f)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1321 ]
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1322
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1323 if not fastpathlinkrev:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1324
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1325 def normallinknodes(unused, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1326 return fnodes.get(fname, {})
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1327
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1328 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1329 cln = self._repo.changelog.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1330
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1331 def normallinknodes(store, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1332 flinkrev = store.linkrev
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1333 fnode = store.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1334 revs = ((r, flinkrev(r)) for r in store)
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44349
diff changeset
1335 return {fnode(r): cln(lr) for r, lr in revs if lr in clrevs}
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1336
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1337 clrevtolocalrev = {}
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1338
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1339 if self._isshallow:
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1340 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1341 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1342 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1343 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1344 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1345
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1346 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1347 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1348 try:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1349 fnode = c.filenode(fname)
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1350 clrevtolocalrev[c.rev()] = flog.rev(fnode)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1351 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1352 pass
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1353 links = normallinknodes(flog, fname)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1354 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1355 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1356 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1357 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1358 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1359 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1360 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1361 return links
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1362
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1363 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1364 linknodes = normallinknodes
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1365
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1366 repo = self._repo
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1367 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1368 _(b'files'), unit=_(b'files'), total=len(changedfiles)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1369 )
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1370 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1371 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1372 if not filerevlog:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1373 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1374 _(b"empty or missing file data for %s") % fname
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1375 )
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1376
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1377 clrevtolocalrev.clear()
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1378
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1379 linkrevnodes = linknodes(filerevlog, fname)
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1380 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1381 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1382 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1383 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1384
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1385 frev, flr = filerevlog.rev, filerevlog.linkrev
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1386 # Skip sending any filenode we know the client already
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1387 # has. This avoids over-sending files relatively
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1388 # inexpensively, so it's not a problem if we under-filter
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1389 # here.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1390 filenodes = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1391 n for n in linkrevnodes if flr(frev(n)) not in commonrevs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1392 ]
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1393
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1394 if not filenodes:
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1395 continue
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1396
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1397 progress.update(i + 1, item=fname)
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1398
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1399 deltas = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1400 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1401 filerevlog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1402 filenodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1403 False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1404 lookupfilelog,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1405 self._forcedeltaparentprev,
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1406 ellipses=self._ellipses,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1407 clrevtolocalrev=clrevtolocalrev,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1408 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1409 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1410 )
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1411
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1412 yield fname, deltas
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1413
38416
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38388
diff changeset
1414 progress.complete()
19200
4cfdec944edf bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents: 19199
diff changeset
1415
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1416
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1417 def _makecg1packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1418 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1419 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1420 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1421 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1422 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1423 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1424 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1425 fullnodes=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1426 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1427 ):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1428 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1429 d.node, d.p1node, d.p2node, d.linknode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1430 )
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1431
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1432 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1433 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1434 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1435 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1436 b'01',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1437 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1438 manifestsend=b'',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1439 forcedeltaparentprev=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1440 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1441 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1442 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1443 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1444 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1445 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1446
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1447
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1448 def _makecg2packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1449 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1450 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1451 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1452 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1453 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1454 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1455 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1456 fullnodes=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1457 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1458 ):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1459 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1460 d.node, d.p1node, d.p2node, d.basenode, d.linknode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1461 )
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1462
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1463 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1464 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1465 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1466 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1467 b'02',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1468 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1469 manifestsend=b'',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1470 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1471 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1472 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1473 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1474 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1475 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1476
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1477
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1478 def _makecg3packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1479 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1480 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1481 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1482 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1483 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1484 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1485 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1486 fullnodes=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1487 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1488 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1489 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1490 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1491 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1492
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1493 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1494 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1495 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1496 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1497 b'03',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1498 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1499 manifestsend=closechunk(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1500 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1501 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1502 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1503 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1504 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1505 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1506
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1507
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1508 def _makecg4packer(
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1509 repo,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1510 oldmatcher,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1511 matcher,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1512 bundlecaps,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1513 ellipses=False,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1514 shallow=False,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1515 ellipsisroots=None,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1516 fullnodes=None,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1517 remote_sidedata=None,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1518 ):
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1519 # Same header func as cg3. Sidedata is in a separate chunk from the delta to
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1520 # differenciate "raw delta" and sidedata.
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1521 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1522 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1523 )
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1524
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1525 return cgpacker(
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1526 repo,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1527 oldmatcher,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1528 matcher,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1529 b'04',
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1530 builddeltaheader=builddeltaheader,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1531 manifestsend=closechunk(),
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1532 bundlecaps=bundlecaps,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1533 ellipses=ellipses,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1534 shallow=shallow,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1535 ellipsisroots=ellipsisroots,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1536 fullnodes=fullnodes,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1537 remote_sidedata=remote_sidedata,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1538 )
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1539
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1540
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1541 _packermap = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1542 b'01': (_makecg1packer, cg1unpacker),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1543 # cg2 adds support for exchanging generaldelta
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1544 b'02': (_makecg2packer, cg2unpacker),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1545 # cg3 adds support for exchanging revlog flags and treemanifests
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1546 b'03': (_makecg3packer, cg3unpacker),
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1547 # ch4 adds support for exchanging sidedata
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1548 b'04': (_makecg4packer, cg4unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1549 }
23168
a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22971
diff changeset
1550
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1551
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1552 def allsupportedversions(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1553 versions = set(_packermap.keys())
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1554 needv03 = False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1555 if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1556 repo.ui.configbool(b'experimental', b'changegroup3')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1557 or repo.ui.configbool(b'experimental', b'treemanifest')
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
1558 or scmutil.istreemanifest(repo)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1559 ):
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1560 # we keep version 03 because we need to to exchange treemanifest data
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1561 #
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1562 # we also keep vresion 01 and 02, because it is possible for repo to
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1563 # contains both normal and tree manifest at the same time. so using
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1564 # older version to pull data is viable
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1565 #
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1566 # (or even to push subset of history)
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1567 needv03 = True
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1568 has_revlogv2 = requirements.REVLOGV2_REQUIREMENT in repo.requirements
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1569 if not has_revlogv2:
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1570 versions.discard(b'04')
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1571 if not needv03:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1572 versions.discard(b'03')
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1573 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1574
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1575
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1576 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1577 def supportedincomingversions(repo):
30633
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30632
diff changeset
1578 return allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1579
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1580
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1581 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1582 def supportedoutgoingversions(repo):
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1583 versions = allsupportedversions(repo)
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
1584 if scmutil.istreemanifest(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1585 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1586 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1587 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1588 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1589 # support versions 01 and 02.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1590 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1591 versions.discard(b'02')
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
1592 if requirements.NARROW_REQUIREMENT in repo.requirements:
36495
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1593 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1594 # support that for stripping and unbundling to work.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1595 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1596 versions.discard(b'02')
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1597 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1598 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1599 # mark LFS entries with REVIDX_EXTSTORED.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1600 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1601 versions.discard(b'02')
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1602
27752
29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config
Martin von Zweigbergk <martinvonz@google.com>
parents: 27751
diff changeset
1603 return versions
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1604
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1605
33676
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1606 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1607 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1608 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1609 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1610
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1611
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1612 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1613 # Finds the smallest version that it's safe to assume clients of the repo
27931
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1614 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1615 # support changegroup 02.
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1616 versions = supportedoutgoingversions(repo)
46666
f4c325bf80fc requirements: also add a generaldelta constant
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46663
diff changeset
1617 if requirements.GENERALDELTA_REQUIREMENT in repo.requirements:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1618 versions.discard(b'01')
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1619 assert versions
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1620 return min(versions)
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1621
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1622
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1623 def getbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1624 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1625 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1626 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1627 oldmatcher=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1628 matcher=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1629 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1630 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1631 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1632 fullnodes=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1633 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1634 ):
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1635 assert version in supportedoutgoingversions(repo)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1636
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1637 if matcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1638 matcher = matchmod.always()
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1639 if oldmatcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1640 oldmatcher = matchmod.never()
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1641
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1642 if version == b'01' and not matcher.always():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1643 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
1644 b'version 01 changegroups do not support sparse file matchers'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1645 )
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1646
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1647 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1648 raise error.Abort(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1649 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1650 b'ellipsis nodes require at least cg3 on client and server, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1651 b'but negotiated version %s'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1652 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1653 % version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1654 )
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1655
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1656 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1657 # filter those out.
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1658 matcher = repo.narrowmatch(matcher)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1659
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1660 fn = _packermap[version][0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1661 return fn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1662 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1663 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1664 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1665 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1666 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1667 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1668 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1669 fullnodes=fullnodes,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1670 remote_sidedata=remote_sidedata,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1671 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1672
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1673
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1674 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1675 return _packermap[version][1](fh, alg, extras=extras)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1676
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1677
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1678 def _changegroupinfo(repo, nodes, source):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1679 if repo.ui.verbose or source == b'bundle':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1680 repo.ui.status(_(b"%d changesets found\n") % len(nodes))
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1681 if repo.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1682 repo.ui.debug(b"list of changesets:\n")
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1683 for node in nodes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1684 repo.ui.debug(b"%s\n" % hex(node))
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1685
34111
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1686
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1687 def makechangegroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1688 repo, outgoing, version, source, fastpath=False, bundlecaps=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1689 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1690 cgstream = makestream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1691 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1692 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1693 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1694 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1695 fastpath=fastpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1696 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1697 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1698 return getunbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1699 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1700 util.chunkbuffer(cgstream),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1701 None,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1702 {b'clcount': len(outgoing.missing)},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1703 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1704
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1705
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1706 def makestream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1707 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1708 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1709 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1710 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1711 fastpath=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1712 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1713 matcher=None,
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1714 remote_sidedata=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1715 ):
46724
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1716 bundler = getbundler(
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1717 version,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1718 repo,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1719 bundlecaps=bundlecaps,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1720 matcher=matcher,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1721 remote_sidedata=remote_sidedata,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46666
diff changeset
1722 )
34118
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
1723
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1724 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1725 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1726 csets = outgoing.missing
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 44470
diff changeset
1727 heads = outgoing.ancestorsof
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1728 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1729 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1730 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1731 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1732 fastpathlinkrev = fastpath or (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1733 repo.filtername is None and heads == sorted(repo.heads())
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1734 )
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1735
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1736 repo.hook(b'preoutgoing', throw=True, source=source)
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1737 _changegroupinfo(repo, csets, source)
23177
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1738 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1739
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1740
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1741 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1742 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1743 files = 0
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1744 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1745 _(b'files'), unit=_(b'files'), total=expectedfiles
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1746 )
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
1747 for chunkdata in iter(source.filelogheader, {}):
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1748 files += 1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1749 f = chunkdata[b"filename"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1750 repo.ui.debug(b"adding %s revisions\n" % f)
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1751 progress.increment()
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1752 fl = repo.file(f)
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1753 o = len(fl)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 23897
diff changeset
1754 try:
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1755 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1756 if not fl.addgroup(deltas, revmap, trp):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1757 raise error.Abort(_(b"received file revlog group is empty"))
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25624
diff changeset
1758 except error.CensoredBaseError as e:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1759 raise error.Abort(_(b"received delta base is censored: %s") % e)
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1760 revisions += len(fl) - o
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1761 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1762 needs = needfiles[f]
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
1763 for new in pycompat.xrange(o, len(fl)):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1764 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1765 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1766 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1767 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1768 raise error.Abort(_(b"received spurious file revlog entry"))
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1769 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1770 del needfiles[f]
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1771 progress.complete()
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1772
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
1773 for f, needs in pycompat.iteritems(needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1774 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1775 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1776 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1777 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1778 except error.LookupError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1779 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1780 _(b'missing file data for %s:%s - run hg verify')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1781 % (f, hex(n))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1782 )
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1783
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1784 return revisions, files