Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 45812:09735cde6275

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
phases: allow registration and boundary advancement with revision sets The core internals either use revision sets already or can trivially use them. Use the new interface in cg1unpacker.apply to avoid materializing the list of all new nodes as it is normally just a revision range. This avoids about 67 Bytes / changeset on AMD64 in peak RSS. Differential Revision: https://phab.mercurial-scm.org/D9232
author Joerg Sonnenberger <joerg@bec.de>
date Mon, 19 Oct 2020 02:54:12 +0200
parents a5206e71c536
children 5d65e04b6a80
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
1 # changegroup.py - Mercurial changegroup manipulation functions
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 # Copyright 2006 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
4 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9437
diff changeset
6 # GNU General Public License version 2 or any later version.
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3859
diff changeset
7
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
8 from __future__ import absolute_import
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
9
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
10 import os
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
11 import struct
20933
d3775db748a0 localrepo: move the addchangegroup method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20932
diff changeset
12 import weakref
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
13
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
14 from .i18n import _
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
15 from .node import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
16 hex,
38922
ee1ea96cf9c9 changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38874
diff changeset
17 nullid,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
18 nullrev,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
19 short,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
20 )
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
21 from .pycompat import open
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
22
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
23 from . import (
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
24 error,
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
25 match as matchmod,
25921
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
26 mdiff,
74b303a637bc changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25831
diff changeset
27 phases,
30945
82f1ef8b4477 py3: convert the mode argument of os.fdopen to unicodes (2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30743
diff changeset
28 pycompat,
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
29 requirements,
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
30 scmutil,
42823
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
31 util,
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
32 )
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42363
diff changeset
33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
34 from .interfaces import repository
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
35
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
36 _CHANGEGROUPV1_DELTA_HEADER = struct.Struct(b"20s20s20s20s")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
37 _CHANGEGROUPV2_DELTA_HEADER = struct.Struct(b"20s20s20s20s20s")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
38 _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(b">20s20s20s20s20sH")
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
39
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
40 LFS_REQUIREMENT = b'lfs'
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
41
35754
fb0be099063f util: move 'readexactly' in the util module
Boris Feld <boris.feld@octobus.net>
parents: 35051
diff changeset
42 readexactly = util.readexactly
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
43
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
44
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
45 def getchunk(stream):
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
46 """return the next chunk from stream as a string"""
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
47 d = readexactly(stream, 4)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
48 l = struct.unpack(b">l", d)[0]
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
49 if l <= 4:
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
50 if l:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
51 raise error.Abort(_(b"invalid chunk length %d") % l)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
52 return b""
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
53 return readexactly(stream, l - 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
54
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
55
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 3932
diff changeset
56 def chunkheader(length):
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
57 """return a changegroup chunk header (string)"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
58 return struct.pack(b">l", length + 4)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
59
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
60
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
61 def closechunk():
9437
1c4e4004f3a6 Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents: 9087
diff changeset
62 """return a changegroup chunk header (string) for a zero-length chunk"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
63 return struct.pack(b">l", 0)
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
64
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
65
39019
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
66 def _fileheader(path):
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
67 """Obtain a changegroup chunk header for a named path."""
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
68 return chunkheader(len(path)) + path
227ebd88ce5e changegroup: pull _fileheader out of cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39018
diff changeset
69
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
70
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
71 def writechunks(ui, chunks, filename, vfs=None):
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
72 """Write chunks to a file and return its filename.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
73
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
74 The stream is assumed to be a bundle file.
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
75 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
76 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
77 """
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
78 fh = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
79 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
80 try:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
81 if filename:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
82 if vfs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
83 fh = vfs.open(filename, b"wb")
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
84 else:
30212
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
85 # Increase default buffer size because default is usually
260af19891f2 changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30211
diff changeset
86 # small (4k is common on Linux).
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
87 fh = open(filename, b"wb", 131072)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
88 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
89 fd, filename = pycompat.mkstemp(prefix=b"hg-bundle-", suffix=b".hg")
43551
313e3a279828 cleanup: remove pointless r-prefixes on double-quoted strings
Augie Fackler <augie@google.com>
parents: 43538
diff changeset
90 fh = os.fdopen(fd, "wb")
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
91 cleanup = filename
26540
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
92 for c in chunks:
7469067de2ba changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26509
diff changeset
93 fh.write(c)
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
94 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
95 return filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
96 finally:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
97 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
98 fh.close()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
99 if cleanup is not None:
20976
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
100 if filename and vfs:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
101 vfs.unlink(cleanup)
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
102 else:
c20f4898631e changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20966
diff changeset
103 os.unlink(cleanup)
3660
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
104
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
105
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
106 class cg1unpacker(object):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
107 """Unpacker for cg1 changegroup streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
108
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
109 A changegroup unpacker handles the framing of the revision data in
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
110 the wire format. Most consumers will want to use the apply()
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
111 method to add the changes from the changegroup to a repository.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
112
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
113 If you're forwarding a changegroup unmodified to another consumer,
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
114 use getchunks(), which returns an iterator of changegroup
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
115 chunks. This is mostly useful for cases where you need to know the
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
116 data stream has ended by observing the end of the changegroup.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
117
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
118 deltachunk() is useful only if you're applying delta data. Most
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
119 consumers should prefer apply() instead.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
120
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
121 A few other public methods exist. Those are used only for
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
122 bundlerepo and some debug commands - their use is discouraged.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
123 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
124
22390
e2806b8613ca changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents: 22070
diff changeset
125 deltaheader = _CHANGEGROUPV1_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
126 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
127 version = b'01'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
128 _grouplistcount = 1 # One list of files after the manifests
27920
da5f23362517 changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27867
diff changeset
129
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
130 def __init__(self, fh, alg, extras=None):
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
131 if alg is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
132 alg = b'UN'
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
133 if alg not in util.compengines.supportedbundletypes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
134 raise error.Abort(_(b'unknown stream compression type: %s') % alg)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
135 if alg == b'BZ':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
136 alg = b'_truncatedBZ'
30364
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
137
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
138 compengine = util.compengines.forbundletype(alg)
a37a96d838b9 changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30349
diff changeset
139 self._stream = compengine.decompressorreader(fh)
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
140 self._type = alg
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
141 self.extras = extras or {}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
142 self.callback = None
26706
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
143
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
144 # These methods (compressed, read, seek, tell) all appear to only
8c0c3059f478 changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents: 26704
diff changeset
145 # be used by bundlerepo, but it's a little hard to tell.
12044
bcc7139521b7 bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents: 12043
diff changeset
146 def compressed(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
147 return self._type is not None and self._type != b'UN'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
148
12043
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
149 def read(self, l):
bef5effb3db0 bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents: 12042
diff changeset
150 return self._stream.read(l)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
151
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
152 def seek(self, pos):
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
153 return self._stream.seek(pos)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
154
12330
e527b8635881 bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents: 12329
diff changeset
155 def tell(self):
12332
680fe77ab5b8 bundlerepo: use bundle objects everywhere
Matt Mackall <mpm@selenic.com>
parents: 12330
diff changeset
156 return self._stream.tell()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
157
12347
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
158 def close(self):
6277a9469dff bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents: 12336
diff changeset
159 return self._stream.close()
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
160
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
161 def _chunklength(self):
13459
acbe171c8fbe changegroup: fix typo introduced in 9f2c407caf34
Jim Hague <jim.hague@acm.org>
parents: 13458
diff changeset
162 d = readexactly(self._stream, 4)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
163 l = struct.unpack(b">l", d)[0]
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
164 if l <= 4:
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
165 if l:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
166 raise error.Abort(_(b"invalid chunk length %d") % l)
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
167 return 0
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
168 if self.callback:
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
169 self.callback()
13458
9f2c407caf34 changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents: 13457
diff changeset
170 return l - 4
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
171
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
172 def changelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
173 """v10 does not have a changelog header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
174 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
175
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
176 def manifestheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
177 """v10 does not have a manifest header chunk"""
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
178 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
179
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
180 def filelogheader(self):
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
181 """return the header of the filelogs chunk, v10 only has the filename"""
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
182 l = self._chunklength()
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
183 if not l:
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
184 return {}
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
185 fname = readexactly(self._stream, l)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
186 return {b'filename': fname}
12334
50946802593d bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents: 12333
diff changeset
187
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
188 def _deltaheader(self, headertuple, prevnode):
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
189 node, p1, p2, cs = headertuple
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
190 if prevnode is None:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
191 deltabase = p1
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
192 else:
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
193 deltabase = prevnode
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
194 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
195 return node, p1, p2, deltabase, cs, flags
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
196
14144
3c3c53d8343a unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14143
diff changeset
197 def deltachunk(self, prevnode):
26707
5ee6bd529300 changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents: 26706
diff changeset
198 l = self._chunklength()
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
199 if not l:
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
200 return {}
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
201 headerdata = readexactly(self._stream, self.deltaheadersize)
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
202 header = self.deltaheader.unpack(headerdata)
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 14060
diff changeset
203 delta = readexactly(self._stream, l - self.deltaheadersize)
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
204 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode)
34301
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
205 return (node, p1, p2, cs, deltabase, delta, flags)
12336
9d234f7d8a77 bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents: 12335
diff changeset
206
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
207 def getchunks(self):
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
208 """returns all the chunks contains in the bundle
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
209
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
210 Used when you need to forward the binary stream to a file or another
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
211 network API. To do so, it parse the changegroup data, otherwise it will
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
212 block in case of sshrepo because it don't know the end of the stream.
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
213 """
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
214 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog,
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
215 # and a list of filelogs. For changegroup 3, we expect 4 parts:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
216 # changelog, manifestlog, a list of tree manifestlogs, and a list of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
217 # filelogs.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
218 #
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
219 # Changelog and manifestlog parts are terminated with empty chunks. The
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
220 # tree and file parts are a list of entry sections. Each entry section
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
221 # is a series of chunks terminating in an empty chunk. The list of these
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
222 # entry sections is terminated in yet another empty chunk, so we know
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
223 # we've reached the end of the tree/file list when we reach an empty
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
224 # chunk that was proceeded by no non-empty chunks.
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
225
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
226 parts = 0
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
227 while parts < 2 + self._grouplistcount:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
228 noentries = True
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
229 while True:
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
230 chunk = getchunk(self)
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
231 if not chunk:
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
232 # The first two empty chunks represent the end of the
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
233 # changelog and the manifestlog portions. The remaining
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
234 # empty chunks represent either A) the end of individual
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
235 # tree or file entries in the file list, or B) the end of
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
236 # the entire list. It's the end of the entire list if there
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
237 # were no entries (i.e. noentries is True).
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
238 if parts < 2:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
239 parts += 1
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
240 elif noentries:
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
241 parts += 1
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
242 break
34106
bbdca7e460c0 changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents: 33738
diff changeset
243 noentries = False
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
244 yield chunkheader(len(chunk))
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
245 pos = 0
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
246 while pos < len(chunk):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
247 next = pos + 2 ** 20
20999
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
248 yield chunk[pos:next]
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
249 pos = next
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
250 yield closechunk()
1e28ec9744bf changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20978
diff changeset
251
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
252 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
253 self.callback = prog.increment
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
254 # no need to check for empty manifest group here:
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
255 # if the result of the merge of 1 and 2 is the same in 3 and 4,
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
256 # no new manifest will be created and the manifest group will
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
257 # be empty during the pull
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
258 self.manifestheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
259 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
260 repo.manifestlog.getstorage(b'').addgroup(deltas, revmap, trp)
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
261 prog.complete()
28360
11287888ce4b changegroup: exclude submanifests from manifest progress
Martin von Zweigbergk <martinvonz@google.com>
parents: 28281
diff changeset
262 self.callback = None
26712
04176eaf911b changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents: 26711
diff changeset
263
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
264 def apply(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
265 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
266 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
267 tr,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
268 srctype,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
269 url,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
270 targetphase=phases.draft,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
271 expectedtotal=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
272 ):
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
273 """Add the changegroup returned by source.read() to this repo.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
274 srctype is a string like 'push', 'pull', or 'unbundle'. url is
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
275 the URL of the repo where this changegroup is coming from.
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
276
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
277 Return an integer summarizing the change to this repo:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
278 - nothing changed or no source: 0
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
279 - more heads than before: 1+added heads (2..n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
280 - fewer heads than before: -1-removed heads (-2..-n)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
281 - number of heads stays the same: 1
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
282 """
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
283 repo = repo.unfiltered()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
284
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
285 def csmap(x):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
286 repo.ui.debug(b"add changeset %s\n" % short(x))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
287 return len(cl)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
288
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
289 def revmap(x):
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
290 return cl.rev(x)
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
291
26880
fa7f8b686633 changegroup: fix the scope of a try finally
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26859
diff changeset
292 try:
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
293 # The transaction may already carry source information. In this
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
294 # case we use the top level data. We overwrite the argument
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
295 # because we need to use the top level value (if they exist)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
296 # in this function.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
297 srctype = tr.hookargs.setdefault(b'source', srctype)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
298 tr.hookargs.setdefault(b'url', url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
299 repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
300 b'prechangegroup', throw=True, **pycompat.strkwargs(tr.hookargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
301 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
302
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
303 # write changelog data to temp files so concurrent readers
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
304 # will not see an inconsistent view
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
305 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
306 cl.delayupdate(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
307 oldheads = set(cl.heads())
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
308
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
309 trp = weakref.proxy(tr)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
310 # pull off the changeset group
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
311 repo.ui.status(_(b"adding changesets\n"))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
312 clstart = len(cl)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
313 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
314 _(b'changesets'), unit=_(b'chunks'), total=expectedtotal
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
315 )
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
316 self.callback = progress.increment
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
317
43537
ea0b44255a31 changegroup: avoid shadowing a set with an int
Augie Fackler <augie@google.com>
parents: 43132
diff changeset
318 efilesset = set()
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
319 cgnodes = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
320
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
321 def ondupchangelog(cl, node):
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
322 if cl.rev(node) < clstart:
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
323 cgnodes.append(node)
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
324
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
325 def onchangelog(cl, node):
43537
ea0b44255a31 changegroup: avoid shadowing a set with an int
Augie Fackler <augie@google.com>
parents: 43132
diff changeset
326 efilesset.update(cl.readfiles(node))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
327
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
328 self.changelogheader()
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
329 deltas = self.deltaiter()
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
330 if not cl.addgroup(
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
331 deltas,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
332 csmap,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
333 trp,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
334 addrevisioncb=onchangelog,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
335 duplicaterevisioncb=ondupchangelog,
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
336 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
337 repo.ui.develwarn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
338 b'applied empty changelog from changegroup',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
339 config=b'warn-empty-changegroup',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
340 )
45811
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45729
diff changeset
341 efiles = len(efilesset)
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
342 clend = len(cl)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
343 changesets = clend - clstart
38379
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38352
diff changeset
344 progress.complete()
45729
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
345 del deltas
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
346 # TODO Python 2.7 removal
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
347 # del efilesset
44d84b726c8a unbundle: free temporary objects after use
Joerg Sonnenberger <joerg@bec.de>
parents: 45558
diff changeset
348 efilesset = None
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
349 self.callback = None
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
350
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
351 # pull off the manifest group
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
352 repo.ui.status(_(b"adding manifests\n"))
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
353 # We know that we'll never have more manifests than we had
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
354 # changesets.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
355 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
356 _(b'manifests'), unit=_(b'chunks'), total=changesets
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
357 )
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
358 self._unpackmanifests(repo, revmap, trp, progress)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
359
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
360 needfiles = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
361 if repo.ui.configbool(b'server', b'validate'):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
362 cl = repo.changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
363 ml = repo.manifestlog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
364 # validate incoming csets have their manifests
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
365 for cset in pycompat.xrange(clstart, clend):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
366 mfnode = cl.changelogrevision(cset).manifest
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
367 mfest = ml[mfnode].readdelta()
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
368 # store file nodes we must see
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
369 for f, n in pycompat.iteritems(mfest):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
370 needfiles.setdefault(f, set()).add(n)
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
371
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
372 # process the files
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
373 repo.ui.status(_(b"adding file changes\n"))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
374 newrevs, newfiles = _addchangegroupfiles(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
375 repo, self, revmap, trp, efiles, needfiles
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
376 )
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
377
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
378 # making sure the value exists
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
379 tr.changes.setdefault(b'changegroup-count-changesets', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
380 tr.changes.setdefault(b'changegroup-count-revisions', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
381 tr.changes.setdefault(b'changegroup-count-files', 0)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
382 tr.changes.setdefault(b'changegroup-count-heads', 0)
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
383
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
384 # some code use bundle operation for internal purpose. They usually
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
385 # set `ui.quiet` to do this outside of user sight. Size the report
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
386 # of such operation now happens at the end of the transaction, that
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
387 # ui.quiet has not direct effect on the output.
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
388 #
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
389 # To preserve this intend use an inelegant hack, we fail to report
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
390 # the change if `quiet` is set. We should probably move to
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
391 # something better, but this is a good first step to allow the "end
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
392 # of transaction report" to pass tests.
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
393 if not repo.ui.quiet:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
394 tr.changes[b'changegroup-count-changesets'] += changesets
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
395 tr.changes[b'changegroup-count-revisions'] += newrevs
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
396 tr.changes[b'changegroup-count-files'] += newfiles
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
397
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
398 deltaheads = 0
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
399 if oldheads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
400 heads = cl.heads()
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
401 deltaheads += len(heads) - len(oldheads)
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
402 for h in heads:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
403 if h not in oldheads and repo[h].closesbranch():
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
404 deltaheads -= 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
405
42903
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
406 # see previous comment about checking ui.quiet
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42823
diff changeset
407 if not repo.ui.quiet:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
408 tr.changes[b'changegroup-count-heads'] += deltaheads
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
409 repo.invalidatevolatilesets()
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
410
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
411 if changesets > 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 if b'node' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
413 tr.hookargs[b'node'] = hex(cl.node(clstart))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
414 tr.hookargs[b'node_last'] = hex(cl.node(clend - 1))
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
415 hookargs = dict(tr.hookargs)
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
416 else:
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
417 hookargs = dict(tr.hookargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
418 hookargs[b'node'] = hex(cl.node(clstart))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
419 hookargs[b'node_last'] = hex(cl.node(clend - 1))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
420 repo.hook(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
421 b'pretxnchangegroup',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
422 throw=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
423 **pycompat.strkwargs(hookargs)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
424 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
425
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
426 added = pycompat.xrange(clstart, clend)
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
427 phaseall = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
428 if srctype in (b'push', b'serve'):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
429 # Old servers can not push the boundary themselves.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
430 # New servers won't push the boundary if changeset already
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
431 # exists locally as secret
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
432 #
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
433 # We should not use added here but the list of all change in
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
434 # the bundle
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
435 if repo.publishing():
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
436 targetphase = phaseall = phases.public
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
437 else:
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
438 # closer target phase computation
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
439
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
440 # Those changesets have been pushed from the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
441 # outside, their phases are going to be pushed
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
442 # alongside. Therefor `targetphase` is
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
443 # ignored.
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
444 targetphase = phaseall = phases.draft
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
445 if added:
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
446 phases.registernew(repo, tr, targetphase, [], revs=added)
33456
ae052d04b89e phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents: 33406
diff changeset
447 if phaseall is not None:
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
448 phases.advanceboundary(repo, tr, phaseall, cgnodes, revs=added)
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
449 cgnodes = []
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
450
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
451 if changesets > 0:
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
452
43798
888bd39ed555 lock: pass "success" boolean to _afterlock callbacks
Kyle Lippincott <spectral@google.com>
parents: 43551
diff changeset
453 def runhooks(unused_success):
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
454 # These hooks run when the lock releases, not when the
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
455 # transaction closes. So it's possible for the changelog
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
456 # to have changed since we last saw it.
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
457 if clstart >= len(repo):
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
458 return
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
459
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 repo.hook(b"changegroup", **pycompat.strkwargs(hookargs))
27867
7ced54ebf972 with: use context manager for transaction in changegroup apply
Bryan O'Sullivan <bryano@fb.com>
parents: 27754
diff changeset
461
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
462 for rev in added:
32949
b08431e1b062 changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents: 32948
diff changeset
463 args = hookargs.copy()
45812
09735cde6275 phases: allow registration and boundary advancement with revision sets
Joerg Sonnenberger <joerg@bec.de>
parents: 45811
diff changeset
464 args[b'node'] = hex(cl.node(rev))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
465 del args[b'node_last']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
466 repo.hook(b"incoming", **pycompat.strkwargs(args))
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
467
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
468 newheads = [h for h in repo.heads() if h not in oldheads]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
469 repo.ui.log(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
470 b"incoming",
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
471 b"%d incoming changes - new heads: %s\n",
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
472 len(added),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
473 b', '.join([hex(c[:6]) for c in newheads]),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
474 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
475
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
476 tr.addpostclose(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
477 b'changegroup-runhooks-%020i' % clstart,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
478 lambda tr: repo._afterlock(runhooks),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
479 )
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
480 finally:
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
481 repo.ui.flush()
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
482 # never return 0 here:
32888
b441296f8e9c changegroup: rename "dh" to the clearer "deltaheads"
Martin von Zweigbergk <martinvonz@google.com>
parents: 32887
diff changeset
483 if deltaheads < 0:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
484 ret = deltaheads - 1
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
485 else:
33042
3e102a8dd52c bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32949
diff changeset
486 ret = deltaheads + 1
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33456
diff changeset
487 return ret
26695
1121fced5b20 changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents: 26694
diff changeset
488
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
489 def deltaiter(self):
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
490 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
491 returns an iterator of the deltas in this changegroup
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
492
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
493 Useful for passing to the underlying storage system to be stored.
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
494 """
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
495 chain = None
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
496 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
34301
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
497 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
498 yield chunkdata
05131c963767 changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents: 34298
diff changeset
499 chain = chunkdata[0]
34160
c8b6ed51386b changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34118
diff changeset
500
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
501
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
502 class cg2unpacker(cg1unpacker):
26708
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
503 """Unpacker for cg2 streams.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
504
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
505 cg2 streams add support for generaldelta, so the delta header
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
506 format is slightly different. All other features about the data
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
507 remain the same.
749d913f24b8 changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents: 26707
diff changeset
508 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
509
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
510 deltaheader = _CHANGEGROUPV2_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
511 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
512 version = b'02'
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
513
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
514 def _deltaheader(self, headertuple, prevnode):
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
515 node, p1, p2, deltabase, cs = headertuple
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
516 flags = 0
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
517 return node, p1, p2, deltabase, cs, flags
23181
832b7ef275c8 changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents: 23178
diff changeset
518
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
519
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
520 class cg3unpacker(cg2unpacker):
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
521 """Unpacker for cg3 streams.
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
522
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
523 cg3 streams add support for exchanging treemanifests and revlog
27753
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
524 flags. It adds the revlog flags to the delta header and an empty chunk
d4071cc73f46 changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents: 27752
diff changeset
525 separating manifests and files.
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
526 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
527
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
528 deltaheader = _CHANGEGROUPV3_DELTA_HEADER
38935
271854adc3a6 changegroup: make delta header struct formatters actual structs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38934
diff changeset
529 deltaheadersize = deltaheader.size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
530 version = b'03'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
531 _grouplistcount = 2 # One list of manifests and one list of files
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
532
27433
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
533 def _deltaheader(self, headertuple, prevnode):
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
534 node, p1, p2, deltabase, cs, flags = headertuple
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
535 return node, p1, p2, deltabase, cs, flags
12f727a5b434 changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents: 27432
diff changeset
536
38352
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
537 def _unpackmanifests(self, repo, revmap, trp, prog):
83534c4ec58b changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
538 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog)
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
539 for chunkdata in iter(self.filelogheader, {}):
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
540 # If we get here, there are directory manifests in the changegroup
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
541 d = chunkdata[b"filename"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
542 repo.ui.debug(b"adding %s revisions\n" % d)
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
543 deltas = self.deltaiter()
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
544 if not repo.manifestlog.getstorage(d).addgroup(deltas, revmap, trp):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
545 raise error.Abort(_(b"received dir revlog group is empty"))
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
546
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
547
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
548 class headerlessfixup(object):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
549 def __init__(self, fh, h):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
550 self._h = h
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
551 self._fh = fh
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
552
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
553 def read(self, n):
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
554 if self._h:
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
555 d, self._h = self._h[:n], self._h[n:]
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
556 if len(d) < n:
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
557 d += readexactly(self._fh, n - len(d))
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
558 return d
13457
e74fe15dc7fd changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents: 13456
diff changeset
559 return readexactly(self._fh, n)
12329
7458de933f26 bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents: 12044
diff changeset
560
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
561
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
562 def _revisiondeltatochunks(delta, headerfn):
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
563 """Serialize a revisiondelta to changegroup chunks."""
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
564
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
565 # The captured revision delta may be encoded as a delta against
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
566 # a base revision or as a full revision. The changegroup format
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
567 # requires that everything on the wire be deltas. So for full
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
568 # revisions, we need to invent a header that says to rewrite
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
569 # data.
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
570
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
571 if delta.delta is not None:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
572 prefix, data = b'', delta.delta
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
573 elif delta.basenode == nullid:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
574 data = delta.revision
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
575 prefix = mdiff.trivialdiffheader(len(data))
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
576 else:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
577 data = delta.revision
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
578 prefix = mdiff.replacediffheader(delta.baserevisionsize, len(data))
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
579
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
580 meta = headerfn(delta)
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
581
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
582 yield chunkheader(len(meta) + len(prefix) + len(data))
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
583 yield meta
39054
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
584 if prefix:
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
585 yield prefix
39b8277e2115 changegroup: differentiate between fulltext and diff based deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39053
diff changeset
586 yield data
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
587
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
588
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
589 def _sortnodesellipsis(store, nodes, cl, lookup):
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
590 """Sort nodes for changegroup generation."""
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
591 # Ellipses serving mode.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
592 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
593 # In a perfect world, we'd generate better ellipsis-ified graphs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
594 # for non-changelog revlogs. In practice, we haven't started doing
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
595 # that yet, so the resulting DAGs for the manifestlog and filelogs
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
596 # are actually full of bogus parentage on all the ellipsis
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
597 # nodes. This has the side effect that, while the contents are
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
598 # correct, the individual DAGs might be completely out of whack in
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
599 # a case like 882681bc3166 and its ancestors (back about 10
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
600 # revisions or so) in the main hg repo.
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
601 #
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
602 # The one invariant we *know* holds is that the new (potentially
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
603 # bogus) DAG shape will be valid if we order the nodes in the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
604 # order that they're introduced in dramatis personae by the
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
605 # changelog, so what we do is we sort the non-changelog histories
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
606 # by the order in which they are used by the changelog.
39035
812eec3f89cb changegroup: remove _clnodetorev
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39034
diff changeset
607 key = lambda n: cl.rev(lookup(n))
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
608 return sorted(nodes, key=key)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
609
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
610
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
611 def _resolvenarrowrevisioninfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
612 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
613 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
614 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
615 rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
616 linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
617 linknode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
618 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
619 fullclnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
620 precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
621 ):
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
622 linkparents = precomputedellipsis[linkrev]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
623
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
624 def local(clrev):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
625 """Turn a changelog revnum into a local revnum.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
626
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
627 The ellipsis dag is stored as revnums on the changelog,
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
628 but when we're producing ellipsis entries for
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
629 non-changelog revlogs, we need to turn those numbers into
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
630 something local. This does that for us, and during the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
631 changelog sending phase will also expand the stored
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
632 mappings as needed.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
633 """
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
634 if clrev == nullrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
635 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
636
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
637 if ischangelog:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
638 return clrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
639
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
640 # Walk the ellipsis-ized changelog breadth-first looking for a
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
641 # change that has been linked from the current revlog.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
642 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
643 # For a flat manifest revlog only a single step should be necessary
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
644 # as all relevant changelog entries are relevant to the flat
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
645 # manifest.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
646 #
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
647 # For a filelog or tree manifest dirlog however not every changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
648 # entry will have been relevant, so we need to skip some changelog
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
649 # nodes even after ellipsis-izing.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
650 walk = [clrev]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
651 while walk:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
652 p = walk[0]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
653 walk = walk[1:]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
654 if p in clrevtolocalrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
655 return clrevtolocalrev[p]
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
656 elif p in fullclnodes:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
657 walk.extend([pp for pp in cl.parentrevs(p) if pp != nullrev])
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
658 elif p in precomputedellipsis:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
659 walk.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
660 [pp for pp in precomputedellipsis[p] if pp != nullrev]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
661 )
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
662 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
663 # In this case, we've got an ellipsis with parents
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
664 # outside the current bundle (likely an
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
665 # incremental pull). We "know" that we can use the
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
666 # value of this same revlog at whatever revision
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
667 # is pointed to by linknode. "Know" is in scare
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
668 # quotes because I haven't done enough examination
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
669 # of edge cases to convince myself this is really
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
670 # a fact - it works for all the (admittedly
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
671 # thorough) cases in our testsuite, but I would be
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
672 # somewhat unsurprised to find a case in the wild
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
673 # where this breaks down a bit. That said, I don't
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
674 # know if it would hurt anything.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
675 for i in pycompat.xrange(rev, 0, -1):
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
676 if store.linkrev(i) == clrev:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
677 return i
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
678 # We failed to resolve a parent for this node, so
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
679 # we crash the changegroup construction.
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
680 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
681 b'unable to resolve parent while packing %r %r'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
682 b' for changeset %r' % (store.indexfile, rev, clrev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
683 )
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
684
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
685 return nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
686
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
687 if not linkparents or (store.parentrevs(rev) == (nullrev, nullrev)):
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
688 p1, p2 = nullrev, nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
689 elif len(linkparents) == 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
690 (p1,) = sorted(local(p) for p in linkparents)
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
691 p2 = nullrev
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
692 else:
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
693 p1, p2 = sorted(local(p) for p in linkparents)
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
694
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
695 p1node, p2node = store.node(p1), store.node(p2)
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
696
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
697 return p1node, p2node, linknode
39042
e11d07cc125c changegroup: make _revisiondeltanarrow() a standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39041
diff changeset
698
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
699
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
700 def deltagroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
701 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
702 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
703 nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
704 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
705 lookup,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
706 forcedeltaparentprev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
707 topic=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
708 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
709 clrevtolocalrev=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
710 fullclnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
711 precomputedellipsis=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
712 ):
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
713 """Calculate deltas for a set of revisions.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
714
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
715 Is a generator of ``revisiondelta`` instances.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
716
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
717 If topic is not None, progress detail will be generated using this
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
718 topic name (e.g. changesets, manifests, etc).
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
719 """
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
720 if not nodes:
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
721 return
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
722
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
723 cl = repo.changelog
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
724
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
725 if ischangelog:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
726 # `hg log` shows changesets in storage order. To preserve order
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
727 # across clones, send out changesets in storage order.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
728 nodesorder = b'storage'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
729 elif ellipses:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
730 nodes = _sortnodesellipsis(store, nodes, cl, lookup)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
731 nodesorder = b'nodes'
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
732 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
733 nodesorder = None
39257
2646b8d66b7b changegroup: move node sorting into deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39214
diff changeset
734
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
735 # Perform ellipses filtering and revision massaging. We do this before
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
736 # emitrevisions() because a) filtering out revisions creates less work
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
737 # for emitrevisions() b) dropping revisions would break emitrevisions()'s
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
738 # assumptions about delta choices and we would possibly send a delta
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
739 # referencing a missing base revision.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
740 #
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
741 # Also, calling lookup() has side-effects with regards to populating
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
742 # data structures. If we don't call lookup() for each node or if we call
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
743 # lookup() after the first pass through each node, things can break -
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
744 # possibly intermittently depending on the python hash seed! For that
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
745 # reason, we store a mapping of all linknodes during the initial node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
746 # pass rather than use lookup() on the output side.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
747 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
748 filtered = []
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
749 adjustedparents = {}
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
750 linknodes = {}
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
751
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
752 for node in nodes:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
753 rev = store.rev(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
754 linknode = lookup(node)
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
755 linkrev = cl.rev(linknode)
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
756 clrevtolocalrev[linkrev] = rev
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
757
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
758 # If linknode is in fullclnodes, it means the corresponding
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
759 # changeset was a full changeset and is being sent unaltered.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
760 if linknode in fullclnodes:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
761 linknodes[node] = linknode
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
762
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
763 # If the corresponding changeset wasn't in the set computed
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
764 # as relevant to us, it should be dropped outright.
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
765 elif linkrev not in precomputedellipsis:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
766 continue
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
767
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
768 else:
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
769 # We could probably do this later and avoid the dict
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
770 # holding state. But it likely doesn't matter.
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
771 p1node, p2node, linknode = _resolvenarrowrevisioninfo(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
772 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
773 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
774 ischangelog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
775 rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
776 linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
777 linknode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
778 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
779 fullclnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
780 precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
781 )
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
782
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
783 adjustedparents[node] = (p1node, p2node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
784 linknodes[node] = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
785
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
786 filtered.append(node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
787
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
788 nodes = filtered
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
789
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
790 # We expect the first pass to be fast, so we only engage the progress
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
791 # meter for constructing the revision deltas.
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
792 progress = None
39267
0617a700ef7b changegroup: change topics during generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39266
diff changeset
793 if topic is not None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
794 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
795 topic, unit=_(b'chunks'), total=len(nodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
796 )
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
797
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
798 configtarget = repo.ui.config(b'devel', b'bundle.delta')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
799 if configtarget not in (b'', b'p1', b'full'):
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
800 msg = _("""config "devel.bundle.delta" as unknown value: %s""")
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
801 repo.ui.warn(msg % configtarget)
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
802
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
803 deltamode = repository.CG_DELTAMODE_STD
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
804 if forcedeltaparentprev:
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
805 deltamode = repository.CG_DELTAMODE_PREV
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
806 elif configtarget == b'p1':
40497
968dd7e02ac5 changegroup: allow to force delta to be against p1
Boris Feld <boris.feld@octobus.net>
parents: 40496
diff changeset
807 deltamode = repository.CG_DELTAMODE_P1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
808 elif configtarget == b'full':
40498
808b762679cd changegroup: add a option to create bundle with full snapshot only
Boris Feld <boris.feld@octobus.net>
parents: 40497
diff changeset
809 deltamode = repository.CG_DELTAMODE_FULL
40496
6a917075535a storage: also use `deltamode argument` for ifiledata
Boris Feld <boris.feld@octobus.net>
parents: 40344
diff changeset
810
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
811 revisions = store.emitrevisions(
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
812 nodes,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
813 nodesorder=nodesorder,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
814 revisiondata=True,
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
815 assumehaveparentrevisions=not ellipses,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
816 deltamode=deltamode,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
817 )
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
818
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
819 for i, revision in enumerate(revisions):
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
820 if progress:
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
821 progress.update(i + 1)
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
822
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
823 if ellipses:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
824 linknode = linknodes[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
825
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
826 if revision.node in adjustedparents:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
827 p1node, p2node = adjustedparents[revision.node]
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
828 revision.p1node = p1node
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
829 revision.p2node = p2node
40048
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39871
diff changeset
830 revision.flags |= repository.REVISION_FLAG_ELLIPSIS
39870
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
831
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
832 else:
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
833 linknode = lookup(revision.node)
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
834
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
835 revision.linknode = linknode
31b7e8e7132e changegroup: port to emitrevisions() (issue5976)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39866
diff changeset
836 yield revision
39056
e793e11e1462 changegroup: introduce requests to define delta generation
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39055
diff changeset
837
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
838 if progress:
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
839 progress.complete()
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
840
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
841
38941
4c99c6d1ef95 changegroup: rename cg1packer to cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38940
diff changeset
842 class cgpacker(object):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
843 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
844 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
845 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
846 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
847 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
848 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
849 builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
850 manifestsend,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
851 forcedeltaparentprev=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
852 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
853 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
854 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
855 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
856 fullnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
857 ):
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
858 """Given a source repo, construct a bundler.
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
859
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
860 oldmatcher is a matcher that matches on files the client already has.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
861 These will not be included in the changegroup.
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
862
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
863 matcher is a matcher that matches on files to include in the
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
864 changegroup. Used to facilitate sparse changegroups.
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
865
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
866 forcedeltaparentprev indicates whether delta parents must be against
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
867 the previous revision in a delta group. This should only be used for
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
868 compatibility with changegroup version 1.
38940
23ae0c07a3e1 changegroup: control delta parent behavior via constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38939
diff changeset
869
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
870 builddeltaheader is a callable that constructs the header for a group
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
871 delta.
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
872
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
873 manifestsend is a chunk to send after manifests have been fully emitted.
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
874
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
875 ellipses indicates whether ellipsis serving mode is enabled.
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
876
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
877 bundlecaps is optional and can be used to specify the set of
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
878 capabilities which can be used to build the bundle. While bundlecaps is
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
879 unused in core Mercurial, extensions rely on this feature to communicate
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
880 capabilities to customize the changegroup packer.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
881
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
882 shallow indicates whether shallow data might be sent. The packer may
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
883 need to pack file contents not introduced by the changes being packed.
38948
1af339c22aeb changegroup: move fullnodes into cgpacker
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38947
diff changeset
884
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
885 fullnodes is the set of changelog nodes which should not be ellipsis
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
886 nodes. We store this rather than the set of nodes that should be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
887 ellipsis because for very large histories we expect this to be
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
888 significantly smaller.
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
889 """
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
890 assert oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
891 assert matcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
892 self._oldmatcher = oldmatcher
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
893 self._matcher = matcher
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
894
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
895 self.version = version
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
896 self._forcedeltaparentprev = forcedeltaparentprev
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
897 self._builddeltaheader = builddeltaheader
38937
67f37e8a5490 changegroup: pass end of manifests marker into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38936
diff changeset
898 self._manifestsend = manifestsend
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
899 self._ellipses = ellipses
38934
d7ac49c2353c changegroup: pass version into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38933
diff changeset
900
32327
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
901 # Set of capabilities we can use to build the bundle.
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
902 if bundlecaps is None:
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
903 bundlecaps = set()
df3cf9422e1b changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents: 32308
diff changeset
904 self._bundlecaps = bundlecaps
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
905 self._isshallow = shallow
39034
5baafb8fe253 changegroup: rename _fullnodes to _fullclnodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39033
diff changeset
906 self._fullclnodes = fullnodes
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
907
38946
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
908 # Maps ellipsis revs to their roots at the changelog level.
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
909 self._precomputedellipsis = ellipsisroots
ad4c4cc9a5ac changegroup: pass ellipsis roots into cgpacker constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38945
diff changeset
910
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
911 self._repo = repo
38939
6e999a2d8fe7 changegroup: control reordering via constructor argument
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38938
diff changeset
912
23748
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
913 if self._repo.ui.verbose and not self._repo.ui.debugflag:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
914 self._verbosenote = self._repo.ui.note
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
915 else:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
916 self._verbosenote = lambda s: None
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 23382
diff changeset
917
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
918 def generate(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
919 self, commonrevs, clnodes, fastpathlinkrev, source, changelog=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
920 ):
39688
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
921 """Yield a sequence of changegroup byte chunks.
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
922 If changelog is False, changelog data won't be added to changegroup
a1942015c10e changegroup: add functionality to skip adding changelog data to changegroup
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39686
diff changeset
923 """
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
924
19202
0455fc94ae00 bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents: 19201
diff changeset
925 repo = self._repo
24978
f52560c64953 changegroup: drop _changelog and _manifest properties
Martin von Zweigbergk <martinvonz@google.com>
parents: 24977
diff changeset
926 cl = repo.changelog
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
927
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
928 self._verbosenote(_(b'uncompressed size of bundle content:\n'))
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
929 size = 0
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
930
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
931 clstate, deltas = self._generatechangelog(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
932 cl, clnodes, generate=changelog
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
933 )
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
934 for delta in deltas:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
935 for chunk in _revisiondeltatochunks(delta, self._builddeltaheader):
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
936 size += len(chunk)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
937 yield chunk
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
938
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
939 close = closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
940 size += len(close)
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
941 yield closechunk()
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
942
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
943 self._verbosenote(_(b'%8.i (changelog)\n') % size)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
944
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
945 clrevorder = clstate[b'clrevorder']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
946 manifests = clstate[b'manifests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
947 changedfiles = clstate[b'changedfiles']
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
948
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
949 # We need to make sure that the linkrev in the changegroup refers to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
950 # the first changeset that introduced the manifest or file revision.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
951 # The fastpath is usually safer than the slowpath, because the filelogs
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
952 # are walked in revlog order.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
953 #
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
954 # When taking the slowpath when the manifest revlog uses generaldelta,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
955 # the manifest may be walked in the "wrong" order. Without 'clrevorder',
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
956 # we would get an incorrect linkrev (see fix in cc0ff93d0c0c).
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
957 #
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
958 # When taking the fastpath, we are only vulnerable to reordering
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
959 # of the changelog itself. The changelog never uses generaldelta and is
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
960 # never reordered. To handle this case, we simply take the slowpath,
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
961 # which already has the 'clrevorder' logic. This was also fixed in
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
962 # cc0ff93d0c0c.
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
963
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
964 # Treemanifests don't work correctly with fastpathlinkrev
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
965 # either, because we don't discover which directory nodes to
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
966 # send along with files. This could probably be fixed.
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
967 fastpathlinkrev = fastpathlinkrev and not scmutil.istreemanifest(repo)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
968
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
969 fnodes = {} # needed file nodes
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
970
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
971 size = 0
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
972 it = self.generatemanifests(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
973 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
974 clrevorder,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
975 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
976 manifests,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
977 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
978 source,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
979 clstate[b'clrevtomanifestrev'],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
980 )
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
981
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
982 for tree, deltas in it:
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
983 if tree:
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
984 assert self.version == b'03'
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
985 chunk = _fileheader(tree)
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
986 size += len(chunk)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
987 yield chunk
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
988
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
989 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
990 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
991 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
992 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
993 yield chunk
39050
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
994
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
995 close = closechunk()
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
996 size += len(close)
c921ad9cae08 changegroup: move manifest chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39049
diff changeset
997 yield close
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
998
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
999 self._verbosenote(_(b'%8.i (manifests)\n') % size)
39049
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
1000 yield self._manifestsend
2ebdd265fe8c changegroup: move size tracking and end of manifests to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39048
diff changeset
1001
39021
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
1002 mfdicts = None
fbbda9ff3deb changegroup: pass mfdicts properly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39020
diff changeset
1003 if self._ellipses and self._isshallow:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1004 mfdicts = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1005 (self._repo.manifestlog[n].read(), lr)
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
1006 for (n, lr) in pycompat.iteritems(manifests)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1007 ]
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1008
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1009 manifests.clear()
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44349
diff changeset
1010 clrevs = {cl.rev(x) for x in clnodes}
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1011
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1012 it = self.generatefiles(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1013 changedfiles,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1014 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1015 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1016 mfdicts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1017 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1018 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1019 clrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1020 )
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1021
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1022 for path, deltas in it:
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1023 h = _fileheader(path)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1024 size = len(h)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1025 yield h
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1026
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1027 for delta in deltas:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1028 chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1029 for chunk in chunks:
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1030 size += len(chunk)
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1031 yield chunk
39051
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1032
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1033 close = closechunk()
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1034 size += len(close)
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1035 yield close
c4a2d19d393a changegroup: move file chunk emission to generate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39050
diff changeset
1036
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1037 self._verbosenote(_(b'%8.i %s\n') % (size, path))
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1038
39040
eb8a0139ace3 changegroup: inline _close()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39039
diff changeset
1039 yield closechunk()
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1040
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1041 if clnodes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1042 repo.hook(b'outgoing', node=hex(clnodes[0]), source=source)
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1043
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1044 def _generatechangelog(self, cl, nodes, generate=True):
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1045 """Generate data for changelog chunks.
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1046
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1047 Returns a 2-tuple of a dict containing state and an iterable of
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1048 byte chunks. The state will not be fully populated until the
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1049 chunk stream has been fully consumed.
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1050
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1051 if generate is False, the state will be fully populated and no chunk
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1052 stream will be yielded
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1053 """
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1054 clrevorder = {}
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1055 manifests = {}
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1056 mfl = self._repo.manifestlog
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1057 changedfiles = set()
39036
40374b4a780f changegroup: track changelog to manifest revision map explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39035
diff changeset
1058 clrevtomanifestrev = {}
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1059
41467
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1060 state = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1061 b'clrevorder': clrevorder,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1062 b'manifests': manifests,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1063 b'changedfiles': changedfiles,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1064 b'clrevtomanifestrev': clrevtomanifestrev,
41467
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1065 }
73a33fe625bb changegroup: initialize the state variable a bit earlier
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41387
diff changeset
1066
41468
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1067 if not (generate or self._ellipses):
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1068 # sort the nodes in storage order
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1069 nodes = sorted(nodes, key=cl.rev)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1070 for node in nodes:
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1071 c = cl.changelogrevision(node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1072 clrevorder[node] = len(clrevorder)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1073 # record the first changeset introducing this manifest version
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1074 manifests.setdefault(c.manifest, node)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1075 # Record a complete list of potentially-changed files in
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1076 # this manifest.
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1077 changedfiles.update(c.files)
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1078
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1079 return state, ()
fa7d61f9c512 changegroup: don't try to build changelog chunks if not required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41467
diff changeset
1080
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1081 # Callback for the changelog, used to collect changed files and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1082 # manifest nodes.
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1083 # Returns the linkrev node (identity in the changelog case).
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1084 def lookupcl(x):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1085 c = cl.changelogrevision(x)
23381
cc0ff93d0c0c changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents: 23226
diff changeset
1086 clrevorder[x] = len(clrevorder)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1087
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1088 if self._ellipses:
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1089 # Only update manifests if x is going to be sent. Otherwise we
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1090 # end up with bogus linkrevs specified for manifests and
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1091 # we skip some manifest nodes that we should otherwise
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1092 # have sent.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1093 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1094 x in self._fullclnodes
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1095 or cl.rev(x) in self._precomputedellipsis
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1096 ):
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1097
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1098 manifestnode = c.manifest
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1099 # Record the first changeset introducing this manifest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1100 # version.
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1101 manifests.setdefault(manifestnode, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1102 # Set this narrow-specific dict so we have the lowest
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1103 # manifest revnum to look up for this cl revnum. (Part of
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1104 # mapping changelog ellipsis parents to manifest ellipsis
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1105 # parents)
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1106 clrevtomanifestrev.setdefault(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1107 cl.rev(x), mfl.rev(manifestnode)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1108 )
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1109 # We can't trust the changed files list in the changeset if the
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1110 # client requested a shallow clone.
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1111 if self._isshallow:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1112 changedfiles.update(mfl[c.manifest].read().keys())
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1113 else:
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1114 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1115 else:
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1116 # record the first changeset introducing this manifest version
39266
cc957b9dc335 changegroup: rename mfs to manifests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39265
diff changeset
1117 manifests.setdefault(c.manifest, x)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1118 # Record a complete list of potentially-changed files in
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1119 # this manifest.
39265
3634ed953a42 changegroup: clean up changelog callback
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39264
diff changeset
1120 changedfiles.update(c.files)
38929
d706c77449f9 changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38928
diff changeset
1121
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1122 return x
19204
e9c5b1c246dc bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19202
diff changeset
1123
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1124 gen = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1125 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1126 cl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1127 nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1128 True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1129 lookupcl,
39055
ef3d3a2f9aa5 changegroup: refactor delta parent code
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39054
diff changeset
1130 self._forcedeltaparentprev,
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1131 ellipses=self._ellipses,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1132 topic=_(b'changesets'),
39047
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1133 clrevtolocalrev={},
9e8eb2b444e5 changegroup: extract cgpacker.group() to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39046
diff changeset
1134 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1135 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1136 )
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1137
39014
f7228c907ef4 changegroup: factor changelog chunk generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39013
diff changeset
1138 return state, gen
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1139
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1140 def generatemanifests(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1141 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1142 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1143 clrevorder,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1144 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1145 manifests,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1146 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1147 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1148 clrevtolocalrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1149 ):
34161
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1150 """Returns an iterator of changegroup chunks containing manifests.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1151
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1152 `source` is unused here, but is used by extensions like remotefilelog to
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1153 change what is sent based in pulls vs pushes, etc.
75cc1f1e11f2 changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents: 34160
diff changeset
1154 """
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1155 repo = self._repo
30308
bce79dfcf5e4 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents: 30282
diff changeset
1156 mfl = repo.manifestlog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1157 tmfnodes = {b'': manifests}
28227
1c36cc8e7870 changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1158
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1159 # Callback for the manifest, used to collect linkrevs for filelog
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1160 # revisions.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1161 # Returns the linkrev node (collected in lookupcl).
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1162 def makelookupmflinknode(tree, nodes):
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1163 if fastpathlinkrev:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1164 assert not tree
43538
5b5e62c24b2e changegroup: suppress pytype error that's wrong
Augie Fackler <augie@google.com>
parents: 43537
diff changeset
1165 return (
44349
a0ec05d93c8e cleanup: re-run black on the codebase
Augie Fackler <augie@google.com>
parents: 43976
diff changeset
1166 manifests.__getitem__
a0ec05d93c8e cleanup: re-run black on the codebase
Augie Fackler <augie@google.com>
parents: 43976
diff changeset
1167 ) # pytype: disable=unsupported-operands
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1168
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1169 def lookupmflinknode(x):
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1170 """Callback for looking up the linknode for manifests.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1171
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1172 Returns the linkrev node for the specified manifest.
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1173
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1174 SIDE EFFECT:
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1175
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1176 1) fclnodes gets populated with the list of relevant
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1177 file nodes if we're not using fastpathlinkrev
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1178 2) When treemanifests are in use, collects treemanifest nodes
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1179 to send
27219
beb60a898dd0 changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents: 27218
diff changeset
1180
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1181 Note that this means manifests must be completely sent to
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1182 the client before you can trust the list of files and
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1183 treemanifests to send.
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1184 """
35051
d80380ba8e7d changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents: 34734
diff changeset
1185 clnode = nodes[x]
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1186 mdata = mfl.get(tree, x).readfast(shallow=True)
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1187 for p, n, fl in mdata.iterentries():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1188 if fl == b't': # subdirectory manifest
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1189 subtree = tree + p + b'/'
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1190 tmfclnodes = tmfnodes.setdefault(subtree, {})
28241
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1191 tmfclnode = tmfclnodes.setdefault(n, clnode)
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1192 if clrevorder[clnode] < clrevorder[tmfclnode]:
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1193 tmfclnodes[n] = clnode
a4286175ecba changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 28240
diff changeset
1194 else:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1195 f = tree + p
28240
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1196 fclnodes = fnodes.setdefault(f, {})
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1197 fclnode = fclnodes.setdefault(n, clnode)
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1198 if clrevorder[clnode] < clrevorder[fclnode]:
1ac8ce137377 changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents: 28232
diff changeset
1199 fclnodes[n] = clnode
27239
65c47779bcb5 changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents: 27238
diff changeset
1200 return clnode
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1201
28231
3faba927dd93 changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28230
diff changeset
1202 return lookupmflinknode
19206
6308896b1d4a bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents: 19204
diff changeset
1203
28232
829d369fc5a8 changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents: 28231
diff changeset
1204 while tmfnodes:
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1205 tree, nodes = tmfnodes.popitem()
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1206
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1207 should_visit = self._matcher.visitdir(tree[:-1])
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1208 if tree and not should_visit:
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1209 continue
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1210
39272
73cf21b2e8a6 manifest: add getstorage() to manifestlog and use it globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39271
diff changeset
1211 store = mfl.getstorage(tree)
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1212
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1213 if not should_visit:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1214 # No nodes to send because this directory is out of
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1215 # the client's view of the repository (probably
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1216 # because of narrow clones). Do this even for the root
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1217 # directory (tree=='')
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1218 prunednodes = []
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1219 else:
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1220 # Avoid sending any manifest nodes we can prove the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1221 # client already has by checking linkrevs. See the
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1222 # related comment in generatefiles().
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1223 prunednodes = self._prunemanifests(store, nodes, commonrevs)
40705
dba590f27c7a changegroup: avoid instantiating storage if we are not using it
Kyle Lippincott <spectral@google.com>
parents: 40498
diff changeset
1224
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1225 if tree and not prunednodes:
39043
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1226 continue
d56a6b78de3b changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39042
diff changeset
1227
39261
8b9b93bf70b1 changegroup: rename dir to tree to avoid shadowing a built-in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39259
diff changeset
1228 lookupfn = makelookupmflinknode(tree, nodes)
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1229
39052
d662959dc881 changegroup: emit revisiondelta instances from deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39051
diff changeset
1230 deltas = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1231 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1232 store,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1233 prunednodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1234 False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1235 lookupfn,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1236 self._forcedeltaparentprev,
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1237 ellipses=self._ellipses,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1238 topic=_(b'manifests'),
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1239 clrevtolocalrev=clrevtolocalrev,
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1240 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1241 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1242 )
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1243
42363
27d6956d386b match: use '' instead of '.' for root directory (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41776
diff changeset
1244 if not self._oldmatcher.visitdir(store.tree[:-1]):
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1245 yield tree, deltas
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1246 else:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1247 # 'deltas' is a generator and we need to consume it even if
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1248 # we are not going to send it because a side-effect is that
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1249 # it updates tmdnodes (via lookupfn)
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1250 for d in deltas:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1251 pass
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1252 if not tree:
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1253 yield tree, []
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1254
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1255 def _prunemanifests(self, store, nodes, commonrevs):
41776
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1256 if not self._ellipses:
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1257 # In non-ellipses case and large repositories, it is better to
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1258 # prevent calling of store.rev and store.linkrev on a lot of
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1259 # nodes as compared to sending some extra data
1c1c4ef8b72e changegroup: move non-pruning of non-ellipsis manifests to _prunemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 41730
diff changeset
1260 return nodes.copy()
39748
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1261 # This is split out as a separate method to allow filtering
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1262 # commonrevs in extension code.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1263 #
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1264 # TODO(augie): this shouldn't be required, instead we should
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1265 # make filtering of revisions to send delegated to the store
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1266 # layer.
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1267 frev, flr = store.rev, store.linkrev
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1268 return [n for n in nodes if flr(frev(n)) not in commonrevs]
e03c1a63155c changegroup: tease out a temporary prune method for manifests
Augie Fackler <augie@google.com>
parents: 39747
diff changeset
1269
24897
5c35a6040352 changegroup: document that 'source' parameter exists for extensions
Martin von Zweigbergk <martinvonz@google.com>
parents: 24896
diff changeset
1270 # The 'source' parameter is useful for extensions
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1271 def generatefiles(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1272 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1273 changedfiles,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1274 commonrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1275 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1276 mfdicts,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1277 fastpathlinkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1278 fnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1279 clrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1280 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1281 changedfiles = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1282 f
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1283 for f in changedfiles
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1284 if self._matcher(f) and not self._oldmatcher(f)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1285 ]
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1286
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1287 if not fastpathlinkrev:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1288
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1289 def normallinknodes(unused, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1290 return fnodes.get(fname, {})
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1291
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1292 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1293 cln = self._repo.changelog.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1294
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1295 def normallinknodes(store, fname):
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1296 flinkrev = store.linkrev
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1297 fnode = store.node
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1298 revs = ((r, flinkrev(r)) for r in store)
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44349
diff changeset
1299 return {fnode(r): cln(lr) for r, lr in revs if lr in clrevs}
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1300
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1301 clrevtolocalrev = {}
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1302
38943
cdb9bc216771 changegroup: declare shallow flag in constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38942
diff changeset
1303 if self._isshallow:
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1304 # In a shallow clone, the linknodes callback needs to also include
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1305 # those file nodes that are in the manifests we sent but weren't
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1306 # introduced by those manifests.
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1307 commonctxs = [self._repo[c] for c in commonrevs]
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1308 clrev = self._repo.changelog.rev
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1309
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1310 def linknodes(flog, fname):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1311 for c in commonctxs:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1312 try:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1313 fnode = c.filenode(fname)
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1314 clrevtolocalrev[c.rev()] = flog.rev(fnode)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1315 except error.ManifestLookupError:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1316 pass
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1317 links = normallinknodes(flog, fname)
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1318 if len(links) != len(mfdicts):
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1319 for mf, lr in mfdicts:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1320 fnode = mf.get(fname, None)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1321 if fnode in links:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1322 links[fnode] = min(links[fnode], lr, key=clrev)
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1323 elif fnode:
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1324 links[fnode] = lr
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1325 return links
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1326
39037
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1327 else:
b83e9c503f2f changegroup: define linknodes callbacks in generatefiles()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39036
diff changeset
1328 linknodes = normallinknodes
38928
a06aab274aef changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38927
diff changeset
1329
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1330 repo = self._repo
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1331 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1332 _(b'files'), unit=_(b'files'), total=len(changedfiles)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1333 )
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1334 for i, fname in enumerate(sorted(changedfiles)):
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1335 filerevlog = repo.file(fname)
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1336 if not filerevlog:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1337 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1338 _(b"empty or missing file data for %s") % fname
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1339 )
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1340
39039
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1341 clrevtolocalrev.clear()
a6e1ff40e335 changegroup: pass clrevtolocalrev to each group
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39038
diff changeset
1342
19334
95a49112e7ab bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents: 19325
diff changeset
1343 linkrevnodes = linknodes(filerevlog, fname)
19207
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1344 # Lookup for filenodes, we collected the linkrev nodes above in the
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1345 # fastpath case and with lookupmf in the slowpath case.
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1346 def lookupfilelog(x):
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1347 return linkrevnodes[x]
a67e1380dfbd bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 19206
diff changeset
1348
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1349 frev, flr = filerevlog.rev, filerevlog.linkrev
39749
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1350 # Skip sending any filenode we know the client already
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1351 # has. This avoids over-sending files relatively
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1352 # inexpensively, so it's not a problem if we under-filter
5adc5fe41a7d changegroup: reintroduce some comments that have gotten lost over the years
Augie Fackler <augie@google.com>
parents: 39748
diff changeset
1353 # here.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1354 filenodes = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1355 n for n in linkrevnodes if flr(frev(n)) not in commonrevs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1356 ]
39045
39f5c7afdc25 changegroup: inline _prune() into call sites
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39044
diff changeset
1357
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1358 if not filenodes:
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1359 continue
39020
037debbf869c changegroup: pass sorted revisions into group() (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39019
diff changeset
1360
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1361 progress.update(i + 1, item=fname)
39046
8c84f1ef949e changegroup: pass all state into group()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39045
diff changeset
1362
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1363 deltas = deltagroup(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1364 self._repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1365 filerevlog,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1366 filenodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1367 False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1368 lookupfilelog,
39866
db5501d93bcf changegroup: remove reordering control (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39749
diff changeset
1369 self._forcedeltaparentprev,
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1370 ellipses=self._ellipses,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1371 clrevtolocalrev=clrevtolocalrev,
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1372 fullclnodes=self._fullclnodes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1373 precomputedellipsis=self._precomputedellipsis,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1374 )
39058
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1375
0b5f534df82a changegroup: invert conditional and dedent
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39057
diff changeset
1376 yield fname, deltas
39048
fcdab6629dde changegroup: emit delta group close chunk outside of deltagroup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39047
diff changeset
1377
38416
1c5c4a5dd86d changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38388
diff changeset
1378 progress.complete()
19200
4cfdec944edf bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents: 19199
diff changeset
1379
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1380
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1381 def _makecg1packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1382 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1383 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1384 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1385 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1386 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1387 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1388 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1389 fullnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1390 ):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1391 builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1392 d.node, d.p1node, d.p2node, d.linknode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1393 )
27432
77d25b913f80 changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents: 27241
diff changeset
1394
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1395 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1396 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1397 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1398 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1399 b'01',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1400 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1401 manifestsend=b'',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1402 forcedeltaparentprev=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1403 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1404 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1405 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1406 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1407 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1408 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1409
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1410
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1411 def _makecg2packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1412 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1413 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1414 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1415 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1416 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1417 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1418 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1419 fullnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1420 ):
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1421 builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1422 d.node, d.p1node, d.p2node, d.basenode, d.linknode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1423 )
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1424
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1425 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1426 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1427 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1428 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1429 b'02',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1430 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1431 manifestsend=b'',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1432 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1433 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1434 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1435 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1436 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1437 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1438
38936
bd64b8b8f0dd changegroup: pass function to build delta header into constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38935
diff changeset
1439
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1440 def _makecg3packer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1441 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1442 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1443 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1444 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1445 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1446 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1447 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1448 fullnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1449 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1450 builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1451 d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1452 )
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1453
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1454 return cgpacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1455 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1456 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1457 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1458 b'03',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1459 builddeltaheader=builddeltaheader,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1460 manifestsend=closechunk(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1461 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1462 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1463 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1464 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1465 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1466 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1467
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1468
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1469 _packermap = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1470 b'01': (_makecg1packer, cg1unpacker),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1471 # cg2 adds support for exchanging generaldelta
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1472 b'02': (_makecg2packer, cg2unpacker),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1473 # cg3 adds support for exchanging revlog flags and treemanifests
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1474 b'03': (_makecg3packer, cg3unpacker),
26709
42733e956887 changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents: 26708
diff changeset
1475 }
23168
a92ba36a1a9d changegroup: add a "packermap" dictionary to track different packer versions
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22971
diff changeset
1476
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1477
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1478 def allsupportedversions(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1479 versions = set(_packermap.keys())
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1480 needv03 = False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1481 if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1482 repo.ui.configbool(b'experimental', b'changegroup3')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1483 or repo.ui.configbool(b'experimental', b'treemanifest')
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
1484 or scmutil.istreemanifest(repo)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1485 ):
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1486 # we keep version 03 because we need to to exchange treemanifest data
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1487 #
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1488 # we also keep vresion 01 and 02, because it is possible for repo to
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1489 # contains both normal and tree manifest at the same time. so using
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1490 # older version to pull data is viable
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1491 #
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1492 # (or even to push subset of history)
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1493 needv03 = True
43132
8f807a83d53b sidedata: use only changegroup3 if sidedata is in use
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1494 if b'exp-sidedata-flag' in repo.requirements:
8f807a83d53b sidedata: use only changegroup3 if sidedata is in use
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1495 needv03 = True
8f807a83d53b sidedata: use only changegroup3 if sidedata is in use
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1496 # don't attempt to use 01/02 until we do sidedata cleaning
8f807a83d53b sidedata: use only changegroup3 if sidedata is in use
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1497 versions.discard(b'01')
8f807a83d53b sidedata: use only changegroup3 if sidedata is in use
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43117
diff changeset
1498 versions.discard(b'02')
43059
4bbc9569e722 changegroup: use positive logic for treemanifest changegroup3 logic
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42903
diff changeset
1499 if not needv03:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1500 versions.discard(b'03')
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1501 return versions
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1502
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1503
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1504 # Changegroup versions that can be applied to the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1505 def supportedincomingversions(repo):
30633
a001cd7296a5 changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30632
diff changeset
1506 return allsupportedversions(repo)
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1507
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1508
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1509 # Changegroup versions that can be created from the repo
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1510 def supportedoutgoingversions(repo):
30632
7ace5304fec5 changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30631
diff changeset
1511 versions = allsupportedversions(repo)
45558
10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45392
diff changeset
1512 if scmutil.istreemanifest(repo):
27928
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1513 # Versions 01 and 02 support only flat manifests and it's just too
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1514 # expensive to convert between the flat manifest and tree manifest on
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1515 # the fly. Since tree manifests are hashed differently, all of history
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1516 # would have to be converted. Instead, we simply don't even pretend to
c0f11347b107 changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27920
diff changeset
1517 # support versions 01 and 02.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1518 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1519 versions.discard(b'02')
45392
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 45391
diff changeset
1520 if requirements.NARROW_REQUIREMENT in repo.requirements:
36495
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1521 # Versions 01 and 02 don't support revlog flags, and we need to
94709406f10d narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36494
diff changeset
1522 # support that for stripping and unbundling to work.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1523 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1524 versions.discard(b'02')
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1525 if LFS_REQUIREMENT in repo.requirements:
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1526 # Versions 01 and 02 don't support revlog flags, and we need to
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1527 # mark LFS entries with REVIDX_EXTSTORED.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1528 versions.discard(b'01')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1529 versions.discard(b'02')
37135
a54113fcc8c9 lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 37087
diff changeset
1530
27752
29cfc474c5fd changegroup3: introduce experimental.changegroup3 boolean config
Martin von Zweigbergk <martinvonz@google.com>
parents: 27751
diff changeset
1531 return versions
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1532
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1533
33676
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1534 def localversion(repo):
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1535 # Finds the best version to use for bundles that are meant to be used
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1536 # locally, such as those from strip and shelve, and temporary bundles.
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1537 return max(supportedoutgoingversions(repo))
91f0677dc920 repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents: 33461
diff changeset
1538
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1539
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1540 def safeversion(repo):
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1541 # Finds the smallest version that it's safe to assume clients of the repo
27931
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1542 # will support. For example, all hg versions that support generaldelta also
1289a122cf3f shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents: 27929
diff changeset
1543 # support changegroup 02.
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1544 versions = supportedoutgoingversions(repo)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1545 if b'generaldelta' in repo.requirements:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1546 versions.discard(b'01')
27929
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1547 assert versions
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1548 return min(versions)
3b2ac2115464 changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27928
diff changeset
1549
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1550
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1551 def getbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1552 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1553 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1554 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1555 oldmatcher=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1556 matcher=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1557 ellipses=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1558 shallow=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1559 ellipsisroots=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1560 fullnodes=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1561 ):
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27946
diff changeset
1562 assert version in supportedoutgoingversions(repo)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1563
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1564 if matcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1565 matcher = matchmod.always()
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1566 if oldmatcher is None:
41687
0531dff73d0b match: delete unused root and cwd arguments from {always,never,exact}() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 41684
diff changeset
1567 oldmatcher = matchmod.never()
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1568
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1569 if version == b'01' and not matcher.always():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1570 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
1571 b'version 01 changegroups do not support sparse file matchers'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1572 )
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1573
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1574 if ellipses and version in (b'01', b'02'):
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1575 raise error.Abort(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1576 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1577 b'ellipsis nodes require at least cg3 on client and server, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1578 b'but negotiated version %s'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1579 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1580 % version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1581 )
38947
1469584ad5fe changegroup: specify ellipses mode explicitly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38946
diff changeset
1582
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1583 # Requested files could include files not in the local store. So
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1584 # filter those out.
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40083
diff changeset
1585 matcher = repo.narrowmatch(matcher)
38834
1d01cf0416a5 changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38823
diff changeset
1586
38933
19344024a8e1 changegroup: define functions for creating changegroup packers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38932
diff changeset
1587 fn = _packermap[version][0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1588 return fn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1589 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1590 oldmatcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1591 matcher,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1592 bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1593 ellipses=ellipses,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1594 shallow=shallow,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1595 ellipsisroots=ellipsisroots,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1596 fullnodes=fullnodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1597 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1598
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1599
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1600 def getunbundler(version, fh, alg, extras=None):
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29371
diff changeset
1601 return _packermap[version][1](fh, alg, extras=extras)
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27739
diff changeset
1602
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1603
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1604 def _changegroupinfo(repo, nodes, source):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1605 if repo.ui.verbose or source == b'bundle':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1606 repo.ui.status(_(b"%d changesets found\n") % len(nodes))
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1607 if repo.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1608 repo.ui.debug(b"list of changesets:\n")
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1609 for node in nodes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1610 repo.ui.debug(b"%s\n" % hex(node))
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1611
34111
f85dfde1731a changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34106
diff changeset
1612
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1613 def makechangegroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1614 repo, outgoing, version, source, fastpath=False, bundlecaps=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1615 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1616 cgstream = makestream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1617 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1618 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1619 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1620 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1621 fastpath=fastpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1622 bundlecaps=bundlecaps,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1623 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1624 return getunbundler(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1625 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1626 util.chunkbuffer(cgstream),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1627 None,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1628 {b'clcount': len(outgoing.missing)},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1629 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1630
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1631
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1632 def makestream(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1633 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1634 outgoing,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1635 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1636 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1637 fastpath=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1638 bundlecaps=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1639 matcher=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1640 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1641 bundler = getbundler(version, repo, bundlecaps=bundlecaps, matcher=matcher)
34118
92f1e2be8ab6 changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
1642
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1643 repo = repo.unfiltered()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1644 commonrevs = outgoing.common
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1645 csets = outgoing.missing
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 44470
diff changeset
1646 heads = outgoing.ancestorsof
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1647 # We go through the fast path if we get told to, or if all (unfiltered
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1648 # heads have been requested (since we then know there all linkrevs will
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1649 # be pulled by the client).
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1650 heads.sort()
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1651 fastpathlinkrev = fastpath or (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1652 repo.filtername is None and heads == sorted(repo.heads())
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1653 )
20925
5174c48ed8d8 localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20675
diff changeset
1654
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1655 repo.hook(b'preoutgoing', throw=True, source=source)
20926
7c1ed40e3325 localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20925
diff changeset
1656 _changegroupinfo(repo, csets, source)
23177
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1657 return bundler.generate(commonrevs, csets, fastpathlinkrev, source)
706547a14b8b changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents: 23168
diff changeset
1658
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1659
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1660 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1661 revisions = 0
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1662 files = 0
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1663 progress = repo.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1664 _(b'files'), unit=_(b'files'), total=expectedfiles
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1665 )
29736
4e7be6e33269 changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29704
diff changeset
1666 for chunkdata in iter(source.filelogheader, {}):
28361
277a22cd8741 changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents: 28360
diff changeset
1667 files += 1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1668 f = chunkdata[b"filename"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1669 repo.ui.debug(b"adding %s revisions\n" % f)
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1670 progress.increment()
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1671 fl = repo.file(f)
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1672 o = len(fl)
24120
a450e0a2ba0a revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents: 23897
diff changeset
1673 try:
34298
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1674 deltas = source.deltaiter()
1db9abf407c5 revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents: 34264
diff changeset
1675 if not fl.addgroup(deltas, revmap, trp):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1676 raise error.Abort(_(b"received file revlog group is empty"))
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25624
diff changeset
1677 except error.CensoredBaseError as e:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1678 raise error.Abort(_(b"received delta base is censored: %s") % e)
27754
a09f143daaf4 changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents: 27753
diff changeset
1679 revisions += len(fl) - o
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1680 if f in needfiles:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1681 needs = needfiles[f]
38823
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38557
diff changeset
1682 for new in pycompat.xrange(o, len(fl)):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1683 n = fl.node(new)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1684 if n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1685 needs.remove(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1686 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1687 raise error.Abort(_(b"received spurious file revlog entry"))
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1688 if not needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1689 del needfiles[f]
38388
daa08d45740f changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38379
diff changeset
1690 progress.complete()
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1691
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
1692 for f, needs in pycompat.iteritems(needfiles):
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1693 fl = repo.file(f)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1694 for n in needs:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1695 try:
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1696 fl.rev(n)
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1697 except error.LookupError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26540
diff changeset
1698 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
1699 _(b'missing file data for %s:%s - run hg verify')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1700 % (f, hex(n))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43059
diff changeset
1701 )
20932
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1702
0ac83e4e4f7c localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20931
diff changeset
1703 return revisions, files