Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/changegroup.py @ 38932:23d582caae30
changegroup: capture revision delta in a data structure
The current changegroup generation code is tightly coupled to
the revlog API. This tight coupling makes it difficult to implement
alternate storage backends without requiring a large surface area
of the revlog API to be exposed. This is not desirable.
In order to support changegroup generation with non-revlog storage,
we'll need to abstract the concept of delta generation.
This commit is the first step down that road. We introduce a
data structure for representing a delta in a changegroup.
The API still leaves a lot to be desired. But at least we now
have separation between data and actions performed on it.
As part of this, we tweak behavior slightly: we no longer
concatenate the delta prefix with the metadata header. Instead,
we track and emit the prefix as a separate chunk. This shouldn't
have any meaningful impact since all the chunks just get sent to
the wire, the compressor, etc.
Because we're introducing a new object, this does add some
overhead to changegroup execution. `hg perfchangegroupchangelog`
on my clone of the Mercurial repo (~40,000 visible revisions in
the changelog) slows down a bit:
! wall 1.268600 comb 1.270000 user 1.270000 sys 0.000000 (best of 8)
! wall 1.419479 comb 1.410000 user 1.410000 sys 0.000000 (best of 8)
With for `hg bundle -t none-v2 -a /dev/null`:
before: real 6.610 secs (user 6.460+0.000 sys 0.140+0.000)
after: real 7.210 secs (user 7.060+0.000 sys 0.140+0.000)
I plan to claw back this regression in future commits. And I may
even do away with this data structure once the refactor is complete.
For now, it makes things easier to comprehend.
Differential Revision: https://phab.mercurial-scm.org/D4075
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Fri, 03 Aug 2018 10:05:26 -0700 |
parents | eb022ce9e505 |
children | 19344024a8e1 |
rev | line source |
---|---|
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
1 # changegroup.py - Mercurial changegroup manipulation functions |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
3 # Copyright 2006 Matt Mackall <mpm@selenic.com> |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
4 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
10263 | 6 # GNU General Public License version 2 or any later version. |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
3859
diff
changeset
|
7 |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
8 from __future__ import absolute_import |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
9 |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
10 import os |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
11 import struct |
20933
d3775db748a0
localrepo: move the addchangegroup method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20932
diff
changeset
|
12 import weakref |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
13 |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
14 from .i18n import _ |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
15 from .node import ( |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
16 hex, |
38922
ee1ea96cf9c9
changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38874
diff
changeset
|
17 nullid, |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
18 nullrev, |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
19 short, |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
20 ) |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
21 |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
22 from .thirdparty import ( |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
23 attr, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
24 ) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
25 |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
26 from . import ( |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
27 dagutil, |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
28 error, |
38846
5742d0428ed9
changegroup: inline prune() logic from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38834
diff
changeset
|
29 manifest, |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
30 match as matchmod, |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
31 mdiff, |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
32 phases, |
30945
82f1ef8b4477
py3: convert the mode argument of os.fdopen to unicodes (2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30743
diff
changeset
|
33 pycompat, |
38874
a232e6744ba3
narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents:
38846
diff
changeset
|
34 repository, |
38922
ee1ea96cf9c9
changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38874
diff
changeset
|
35 revlog, |
25921
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
36 util, |
74b303a637bc
changegroup: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25831
diff
changeset
|
37 ) |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
38 |
37087
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
36843
diff
changeset
|
39 from .utils import ( |
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
36843
diff
changeset
|
40 stringutil, |
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
36843
diff
changeset
|
41 ) |
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
36843
diff
changeset
|
42 |
22390
e2806b8613ca
changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents:
22070
diff
changeset
|
43 _CHANGEGROUPV1_DELTA_HEADER = "20s20s20s20s" |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
44 _CHANGEGROUPV2_DELTA_HEADER = "20s20s20s20s20s" |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
45 _CHANGEGROUPV3_DELTA_HEADER = ">20s20s20s20s20sH" |
14141
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
46 |
37135
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
47 LFS_REQUIREMENT = 'lfs' |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
48 |
35754
fb0be099063f
util: move 'readexactly' in the util module
Boris Feld <boris.feld@octobus.net>
parents:
35051
diff
changeset
|
49 readexactly = util.readexactly |
13457
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
50 |
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
51 def getchunk(stream): |
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
52 """return the next chunk from stream as a string""" |
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
53 d = readexactly(stream, 4) |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
54 l = struct.unpack(">l", d)[0] |
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
55 if l <= 4: |
13458
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
56 if l: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
57 raise error.Abort(_("invalid chunk length %d") % l) |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
58 return "" |
13457
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
59 return readexactly(stream, l - 4) |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
60 |
5368
61462e7d62ed
changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents:
3932
diff
changeset
|
61 def chunkheader(length): |
9437
1c4e4004f3a6
Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents:
9087
diff
changeset
|
62 """return a changegroup chunk header (string)""" |
5368
61462e7d62ed
changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents:
3932
diff
changeset
|
63 return struct.pack(">l", length + 4) |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
64 |
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
65 def closechunk(): |
9437
1c4e4004f3a6
Improve some docstrings relating to changegroups and prepush().
Greg Ward <greg-hg@gerg.ca>
parents:
9087
diff
changeset
|
66 """return a changegroup chunk header (string) for a zero-length chunk""" |
1981
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
67 return struct.pack(">l", 0) |
736b6c96bbbc
make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff
changeset
|
68 |
26540
7469067de2ba
changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26509
diff
changeset
|
69 def writechunks(ui, chunks, filename, vfs=None): |
7469067de2ba
changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26509
diff
changeset
|
70 """Write chunks to a file and return its filename. |
3659
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
71 |
26540
7469067de2ba
changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26509
diff
changeset
|
72 The stream is assumed to be a bundle file. |
3659
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
73 Existing files will not be overwritten. |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
74 If no filename is specified, a temporary file is created. |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
75 """ |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
76 fh = None |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
77 cleanup = None |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
78 try: |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
79 if filename: |
20976
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
80 if vfs: |
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
81 fh = vfs.open(filename, "wb") |
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
82 else: |
30212
260af19891f2
changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30211
diff
changeset
|
83 # Increase default buffer size because default is usually |
260af19891f2
changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30211
diff
changeset
|
84 # small (4k is common on Linux). |
260af19891f2
changegroup: increase write buffer size to 128k
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30211
diff
changeset
|
85 fh = open(filename, "wb", 131072) |
3659
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
86 else: |
38197
aac4be30e250
py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents:
37339
diff
changeset
|
87 fd, filename = pycompat.mkstemp(prefix="hg-bundle-", suffix=".hg") |
36843
5bc7ff103081
py3: use r'' instead of sysstr('') to get around code transformer
Yuya Nishihara <yuya@tcha.org>
parents:
36768
diff
changeset
|
88 fh = os.fdopen(fd, r"wb") |
3659
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
89 cleanup = filename |
26540
7469067de2ba
changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26509
diff
changeset
|
90 for c in chunks: |
7469067de2ba
changegroup: extract the file management part in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26509
diff
changeset
|
91 fh.write(c) |
3659
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
92 cleanup = None |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
93 return filename |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
94 finally: |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
95 if fh is not None: |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
96 fh.close() |
025f68f22ae2
move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents:
2470
diff
changeset
|
97 if cleanup is not None: |
20976
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
98 if filename and vfs: |
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
99 vfs.unlink(cleanup) |
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
100 else: |
c20f4898631e
changegroup: add "vfs" argument to "writebundle()" for relative access via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20966
diff
changeset
|
101 os.unlink(cleanup) |
3660
8500a13ec44b
create a readbundle function
Matt Mackall <mpm@selenic.com>
parents:
3659
diff
changeset
|
102 |
22390
e2806b8613ca
changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents:
22070
diff
changeset
|
103 class cg1unpacker(object): |
26708
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
104 """Unpacker for cg1 changegroup streams. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
105 |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
106 A changegroup unpacker handles the framing of the revision data in |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
107 the wire format. Most consumers will want to use the apply() |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
108 method to add the changes from the changegroup to a repository. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
109 |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
110 If you're forwarding a changegroup unmodified to another consumer, |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
111 use getchunks(), which returns an iterator of changegroup |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
112 chunks. This is mostly useful for cases where you need to know the |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
113 data stream has ended by observing the end of the changegroup. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
114 |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
115 deltachunk() is useful only if you're applying delta data. Most |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
116 consumers should prefer apply() instead. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
117 |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
118 A few other public methods exist. Those are used only for |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
119 bundlerepo and some debug commands - their use is discouraged. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
120 """ |
22390
e2806b8613ca
changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents:
22070
diff
changeset
|
121 deltaheader = _CHANGEGROUPV1_DELTA_HEADER |
14141
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
122 deltaheadersize = struct.calcsize(deltaheader) |
23896
becfecaf9087
changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents:
23895
diff
changeset
|
123 version = '01' |
27920
da5f23362517
changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27867
diff
changeset
|
124 _grouplistcount = 1 # One list of files after the manifests |
da5f23362517
changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27867
diff
changeset
|
125 |
29593
953839de96ab
bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29371
diff
changeset
|
126 def __init__(self, fh, alg, extras=None): |
30364
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
127 if alg is None: |
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
128 alg = 'UN' |
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
129 if alg not in util.compengines.supportedbundletypes: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
130 raise error.Abort(_('unknown stream compression type: %s') |
26266
1e042e31bd0c
changegroup: move all compressions utilities in util
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25921
diff
changeset
|
131 % alg) |
26392
127b59787fd5
changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26272
diff
changeset
|
132 if alg == 'BZ': |
127b59787fd5
changegroup: use a different compression key for BZ in HG10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26272
diff
changeset
|
133 alg = '_truncatedBZ' |
30364
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
134 |
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
135 compengine = util.compengines.forbundletype(alg) |
a37a96d838b9
changegroup: use compression engines API
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30349
diff
changeset
|
136 self._stream = compengine.decompressorreader(fh) |
12044
bcc7139521b7
bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents:
12043
diff
changeset
|
137 self._type = alg |
29593
953839de96ab
bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29371
diff
changeset
|
138 self.extras = extras or {} |
12334
50946802593d
bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents:
12333
diff
changeset
|
139 self.callback = None |
26706
8c0c3059f478
changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents:
26704
diff
changeset
|
140 |
8c0c3059f478
changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents:
26704
diff
changeset
|
141 # These methods (compressed, read, seek, tell) all appear to only |
8c0c3059f478
changegroup: note why a few methods on cg1unpacker exist
Augie Fackler <augie@google.com>
parents:
26704
diff
changeset
|
142 # be used by bundlerepo, but it's a little hard to tell. |
12044
bcc7139521b7
bundlerepo: remove duplication of bundle decompressors
Matt Mackall <mpm@selenic.com>
parents:
12043
diff
changeset
|
143 def compressed(self): |
30594
182cacaa4c32
cg1packer: fix `compressed` method
Stanislau Hlebik <stash@fb.com>
parents:
30364
diff
changeset
|
144 return self._type is not None and self._type != 'UN' |
12043
bef5effb3db0
bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents:
12042
diff
changeset
|
145 def read(self, l): |
bef5effb3db0
bundle: introduce bundle class
Matt Mackall <mpm@selenic.com>
parents:
12042
diff
changeset
|
146 return self._stream.read(l) |
12330
e527b8635881
bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents:
12329
diff
changeset
|
147 def seek(self, pos): |
e527b8635881
bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents:
12329
diff
changeset
|
148 return self._stream.seek(pos) |
e527b8635881
bundle: make unbundle object seekable
Matt Mackall <mpm@selenic.com>
parents:
12329
diff
changeset
|
149 def tell(self): |
12332
680fe77ab5b8
bundlerepo: use bundle objects everywhere
Matt Mackall <mpm@selenic.com>
parents:
12330
diff
changeset
|
150 return self._stream.tell() |
12347
6277a9469dff
bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents:
12336
diff
changeset
|
151 def close(self): |
6277a9469dff
bundlerepo: restore close() method
Matt Mackall <mpm@selenic.com>
parents:
12336
diff
changeset
|
152 return self._stream.close() |
12334
50946802593d
bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents:
12333
diff
changeset
|
153 |
26707
5ee6bd529300
changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents:
26706
diff
changeset
|
154 def _chunklength(self): |
13459
acbe171c8fbe
changegroup: fix typo introduced in 9f2c407caf34
Jim Hague <jim.hague@acm.org>
parents:
13458
diff
changeset
|
155 d = readexactly(self._stream, 4) |
13458
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
156 l = struct.unpack(">l", d)[0] |
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
157 if l <= 4: |
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
158 if l: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
159 raise error.Abort(_("invalid chunk length %d") % l) |
13458
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
160 return 0 |
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
161 if self.callback: |
12334
50946802593d
bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents:
12333
diff
changeset
|
162 self.callback() |
13458
9f2c407caf34
changegroup: don't accept odd chunk headers
Mads Kiilerich <mads@kiilerich.com>
parents:
13457
diff
changeset
|
163 return l - 4 |
12334
50946802593d
bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents:
12333
diff
changeset
|
164 |
14144
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
165 def changelogheader(self): |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
166 """v10 does not have a changelog header chunk""" |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
167 return {} |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
168 |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
169 def manifestheader(self): |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
170 """v10 does not have a manifest header chunk""" |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
171 return {} |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
172 |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
173 def filelogheader(self): |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
174 """return the header of the filelogs chunk, v10 only has the filename""" |
26707
5ee6bd529300
changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents:
26706
diff
changeset
|
175 l = self._chunklength() |
14144
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
176 if not l: |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
177 return {} |
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
178 fname = readexactly(self._stream, l) |
20675
f8d50add83e1
changegroup: move from dict() construction to {} literals
Augie Fackler <raf@durin42.com>
parents:
19708
diff
changeset
|
179 return {'filename': fname} |
12334
50946802593d
bundle: refactor progress callback
Matt Mackall <mpm@selenic.com>
parents:
12333
diff
changeset
|
180 |
14141
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
181 def _deltaheader(self, headertuple, prevnode): |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
182 node, p1, p2, cs = headertuple |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
183 if prevnode is None: |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
184 deltabase = p1 |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
185 else: |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
186 deltabase = prevnode |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
187 flags = 0 |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
188 return node, p1, p2, deltabase, cs, flags |
14141
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
189 |
14144
3c3c53d8343a
unbundler: separate delta and header parsing
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14143
diff
changeset
|
190 def deltachunk(self, prevnode): |
26707
5ee6bd529300
changegroup: mark cg1unpacker.chunklength as private
Augie Fackler <augie@google.com>
parents:
26706
diff
changeset
|
191 l = self._chunklength() |
12336
9d234f7d8a77
bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents:
12335
diff
changeset
|
192 if not l: |
9d234f7d8a77
bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents:
12335
diff
changeset
|
193 return {} |
14141
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
194 headerdata = readexactly(self._stream, self.deltaheadersize) |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
195 header = struct.unpack(self.deltaheader, headerdata) |
bd1cbfe5db5c
bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14060
diff
changeset
|
196 delta = readexactly(self._stream, l - self.deltaheadersize) |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
197 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, prevnode) |
34301
05131c963767
changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents:
34298
diff
changeset
|
198 return (node, p1, p2, cs, deltabase, delta, flags) |
12336
9d234f7d8a77
bundle: move chunk parsing into unbundle class
Matt Mackall <mpm@selenic.com>
parents:
12335
diff
changeset
|
199 |
20999
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
200 def getchunks(self): |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
201 """returns all the chunks contains in the bundle |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
202 |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
203 Used when you need to forward the binary stream to a file or another |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
204 network API. To do so, it parse the changegroup data, otherwise it will |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
205 block in case of sshrepo because it don't know the end of the stream. |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
206 """ |
34106
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
207 # For changegroup 1 and 2, we expect 3 parts: changelog, manifestlog, |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
208 # and a list of filelogs. For changegroup 3, we expect 4 parts: |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
209 # changelog, manifestlog, a list of tree manifestlogs, and a list of |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
210 # filelogs. |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
211 # |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
212 # Changelog and manifestlog parts are terminated with empty chunks. The |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
213 # tree and file parts are a list of entry sections. Each entry section |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
214 # is a series of chunks terminating in an empty chunk. The list of these |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
215 # entry sections is terminated in yet another empty chunk, so we know |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
216 # we've reached the end of the tree/file list when we reach an empty |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
217 # chunk that was proceeded by no non-empty chunks. |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
218 |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
219 parts = 0 |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
220 while parts < 2 + self._grouplistcount: |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
221 noentries = True |
20999
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
222 while True: |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
223 chunk = getchunk(self) |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
224 if not chunk: |
34106
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
225 # The first two empty chunks represent the end of the |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
226 # changelog and the manifestlog portions. The remaining |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
227 # empty chunks represent either A) the end of individual |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
228 # tree or file entries in the file list, or B) the end of |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
229 # the entire list. It's the end of the entire list if there |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
230 # were no entries (i.e. noentries is True). |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
231 if parts < 2: |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
232 parts += 1 |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
233 elif noentries: |
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
234 parts += 1 |
20999
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
235 break |
34106
bbdca7e460c0
changegroup: fix to allow empty manifest parts
Durham Goode <durham@fb.com>
parents:
33738
diff
changeset
|
236 noentries = False |
20999
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
237 yield chunkheader(len(chunk)) |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
238 pos = 0 |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
239 while pos < len(chunk): |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
240 next = pos + 2**20 |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
241 yield chunk[pos:next] |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
242 pos = next |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
243 yield closechunk() |
1e28ec9744bf
changegroup: move chunk extraction into a getchunks method of unbundle10
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20978
diff
changeset
|
244 |
38352
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
245 def _unpackmanifests(self, repo, revmap, trp, prog): |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
246 self.callback = prog.increment |
26712
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
247 # no need to check for empty manifest group here: |
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
248 # if the result of the merge of 1 and 2 is the same in 3 and 4, |
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
249 # no new manifest will be created and the manifest group will |
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
250 # be empty during the pull |
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
251 self.manifestheader() |
34298
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
252 deltas = self.deltaiter() |
38557
0db41eb0a3ac
manifest: define and implement addgroup() on manifestlog
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38416
diff
changeset
|
253 repo.manifestlog.addgroup(deltas, revmap, trp) |
38379
ef692614e601
progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents:
38352
diff
changeset
|
254 prog.complete() |
28360
11287888ce4b
changegroup: exclude submanifests from manifest progress
Martin von Zweigbergk <martinvonz@google.com>
parents:
28281
diff
changeset
|
255 self.callback = None |
26712
04176eaf911b
changegroup: move manifest unpacking into its own method
Augie Fackler <augie@google.com>
parents:
26711
diff
changeset
|
256 |
33308
248d5890c80a
changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33227
diff
changeset
|
257 def apply(self, repo, tr, srctype, url, targetphase=phases.draft, |
248d5890c80a
changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33227
diff
changeset
|
258 expectedtotal=None): |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
259 """Add the changegroup returned by source.read() to this repo. |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
260 srctype is a string like 'push', 'pull', or 'unbundle'. url is |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
261 the URL of the repo where this changegroup is coming from. |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
262 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
263 Return an integer summarizing the change to this repo: |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
264 - nothing changed or no source: 0 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
265 - more heads than before: 1+added heads (2..n) |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
266 - fewer heads than before: -1-removed heads (-2..-n) |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
267 - number of heads stays the same: 1 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
268 """ |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
269 repo = repo.unfiltered() |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
270 def csmap(x): |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
271 repo.ui.debug("add changeset %s\n" % short(x)) |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
272 return len(cl) |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
273 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
274 def revmap(x): |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
275 return cl.rev(x) |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
276 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
277 changesets = files = revisions = 0 |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
278 |
26880
fa7f8b686633
changegroup: fix the scope of a try finally
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26859
diff
changeset
|
279 try: |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
280 # The transaction may already carry source information. In this |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
281 # case we use the top level data. We overwrite the argument |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
282 # because we need to use the top level value (if they exist) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
283 # in this function. |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
284 srctype = tr.hookargs.setdefault('source', srctype) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
285 url = tr.hookargs.setdefault('url', url) |
33697
38fc45721334
changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents:
33461
diff
changeset
|
286 repo.hook('prechangegroup', |
38fc45721334
changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents:
33461
diff
changeset
|
287 throw=True, **pycompat.strkwargs(tr.hookargs)) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
288 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
289 # write changelog data to temp files so concurrent readers |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
290 # will not see an inconsistent view |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
291 cl = repo.changelog |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
292 cl.delayupdate(tr) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
293 oldheads = set(cl.heads()) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
294 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
295 trp = weakref.proxy(tr) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
296 # pull off the changeset group |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
297 repo.ui.status(_("adding changesets\n")) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
298 clstart = len(cl) |
38352
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
299 progress = repo.ui.makeprogress(_('changesets'), unit=_('chunks'), |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
300 total=expectedtotal) |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
301 self.callback = progress.increment |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
302 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
303 efiles = set() |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
304 def onchangelog(cl, node): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
305 efiles.update(cl.readfiles(node)) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
306 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
307 self.changelogheader() |
34298
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
308 deltas = self.deltaiter() |
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
309 cgnodes = cl.addgroup(deltas, csmap, trp, addrevisioncb=onchangelog) |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
310 efiles = len(efiles) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
311 |
33308
248d5890c80a
changegroup: remove option to allow empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33227
diff
changeset
|
312 if not cgnodes: |
33309
69c4493a54f9
changegroup: don't fail on empty changegroup (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33308
diff
changeset
|
313 repo.ui.develwarn('applied empty changegroup', |
34734
3572b2031cec
devel-warn: add 'warn-' to 'devel.empty-changegroup' config
Boris Feld <boris.feld@octobus.net>
parents:
34301
diff
changeset
|
314 config='warn-empty-changegroup') |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
315 clend = len(cl) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
316 changesets = clend - clstart |
38379
ef692614e601
progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents:
38352
diff
changeset
|
317 progress.complete() |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
318 self.callback = None |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
319 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
320 # pull off the manifest group |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
321 repo.ui.status(_("adding manifests\n")) |
38352
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
322 # We know that we'll never have more manifests than we had |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
323 # changesets. |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
324 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'), |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
325 total=changesets) |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
326 self._unpackmanifests(repo, revmap, trp, progress) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
327 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
328 needfiles = {} |
33227
86c9aa1d598f
configitems: register the 'server.validate' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33185
diff
changeset
|
329 if repo.ui.configbool('server', 'validate'): |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
330 cl = repo.changelog |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
331 ml = repo.manifestlog |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
332 # validate incoming csets have their manifests |
38823
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38557
diff
changeset
|
333 for cset in pycompat.xrange(clstart, clend): |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
334 mfnode = cl.changelogrevision(cset).manifest |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
335 mfest = ml[mfnode].readdelta() |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
336 # store file cgnodes we must see |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
337 for f, n in mfest.iteritems(): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
338 needfiles.setdefault(f, set()).add(n) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
339 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
340 # process the files |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
341 repo.ui.status(_("adding file changes\n")) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
342 newrevs, newfiles = _addchangegroupfiles( |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
343 repo, self, revmap, trp, efiles, needfiles) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
344 revisions += newrevs |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
345 files += newfiles |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
346 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
347 deltaheads = 0 |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
348 if oldheads: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
349 heads = cl.heads() |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
350 deltaheads = len(heads) - len(oldheads) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
351 for h in heads: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
352 if h not in oldheads and repo[h].closesbranch(): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
353 deltaheads -= 1 |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
354 htext = "" |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
355 if deltaheads: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
356 htext = _(" (%+d heads)") % deltaheads |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
357 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
358 repo.ui.status(_("added %d changesets" |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
359 " with %d changes to %d files%s\n") |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
360 % (changesets, revisions, files, htext)) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
361 repo.invalidatevolatilesets() |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
362 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
363 if changesets > 0: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
364 if 'node' not in tr.hookargs: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
365 tr.hookargs['node'] = hex(cl.node(clstart)) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
366 tr.hookargs['node_last'] = hex(cl.node(clend - 1)) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
367 hookargs = dict(tr.hookargs) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
368 else: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
369 hookargs = dict(tr.hookargs) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
370 hookargs['node'] = hex(cl.node(clstart)) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
371 hookargs['node_last'] = hex(cl.node(clend - 1)) |
33697
38fc45721334
changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents:
33461
diff
changeset
|
372 repo.hook('pretxnchangegroup', |
38fc45721334
changegroup: wrap some ** expansions in strkwargs
Augie Fackler <augie@google.com>
parents:
33461
diff
changeset
|
373 throw=True, **pycompat.strkwargs(hookargs)) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
374 |
38823
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38557
diff
changeset
|
375 added = [cl.node(r) for r in pycompat.xrange(clstart, clend)] |
33456
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
376 phaseall = None |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
377 if srctype in ('push', 'serve'): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
378 # Old servers can not push the boundary themselves. |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
379 # New servers won't push the boundary if changeset already |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
380 # exists locally as secret |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
381 # |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
382 # We should not use added here but the list of all change in |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
383 # the bundle |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
384 if repo.publishing(): |
33456
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
385 targetphase = phaseall = phases.public |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
386 else: |
33456
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
387 # closer target phase computation |
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
388 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
389 # Those changesets have been pushed from the |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
390 # outside, their phases are going to be pushed |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
391 # alongside. Therefor `targetphase` is |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
392 # ignored. |
33456
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
393 targetphase = phaseall = phases.draft |
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
394 if added: |
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
395 phases.registernew(repo, tr, targetphase, added) |
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
396 if phaseall is not None: |
ae052d04b89e
phases: rework phase movement code in 'cg.apply' to use 'registernew'
Boris Feld <boris.feld@octobus.net>
parents:
33406
diff
changeset
|
397 phases.advanceboundary(repo, tr, phaseall, cgnodes) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
398 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
399 if changesets > 0: |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
400 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
401 def runhooks(): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
402 # These hooks run when the lock releases, not when the |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
403 # transaction closes. So it's possible for the changelog |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
404 # to have changed since we last saw it. |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
405 if clstart >= len(repo): |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
406 return |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
407 |
33738
672ad4f3bb84
changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents:
33697
diff
changeset
|
408 repo.hook("changegroup", **pycompat.strkwargs(hookargs)) |
27867
7ced54ebf972
with: use context manager for transaction in changegroup apply
Bryan O'Sullivan <bryano@fb.com>
parents:
27754
diff
changeset
|
409 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
410 for n in added: |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
411 args = hookargs.copy() |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
412 args['node'] = hex(n) |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
413 del args['node_last'] |
33738
672ad4f3bb84
changegroup: more **kwargs
Augie Fackler <augie@google.com>
parents:
33697
diff
changeset
|
414 repo.hook("incoming", **pycompat.strkwargs(args)) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
415 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
416 newheads = [h for h in repo.heads() |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
417 if h not in oldheads] |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
418 repo.ui.log("incoming", |
36760
2a5024109490
py3: fix int formatting of "incoming changes" log
Yuya Nishihara <yuya@tcha.org>
parents:
36495
diff
changeset
|
419 "%d incoming changes - new heads: %s\n", |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
420 len(added), |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
421 ', '.join([hex(c[:6]) for c in newheads])) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
422 |
32949
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
423 tr.addpostclose('changegroup-runhooks-%020i' % clstart, |
b08431e1b062
changegroup: delete "if True" and reflow
Martin von Zweigbergk <martinvonz@google.com>
parents:
32948
diff
changeset
|
424 lambda tr: repo._afterlock(runhooks)) |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
425 finally: |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
426 repo.ui.flush() |
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
427 # never return 0 here: |
32888
b441296f8e9c
changegroup: rename "dh" to the clearer "deltaheads"
Martin von Zweigbergk <martinvonz@google.com>
parents:
32887
diff
changeset
|
428 if deltaheads < 0: |
33042
3e102a8dd52c
bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
32949
diff
changeset
|
429 ret = deltaheads - 1 |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
430 else: |
33042
3e102a8dd52c
bundle2: record changegroup data in 'op.records' (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
32949
diff
changeset
|
431 ret = deltaheads + 1 |
33461
bb72031f0ea8
changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents:
33456
diff
changeset
|
432 return ret |
26695
1121fced5b20
changegroup: migrate addchangegroup() to forward to cg?unpacker.apply()
Augie Fackler <augie@google.com>
parents:
26694
diff
changeset
|
433 |
34298
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
434 def deltaiter(self): |
34160
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
435 """ |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
436 returns an iterator of the deltas in this changegroup |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
437 |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
438 Useful for passing to the underlying storage system to be stored. |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
439 """ |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
440 chain = None |
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
441 for chunkdata in iter(lambda: self.deltachunk(chain), {}): |
34301
05131c963767
changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents:
34298
diff
changeset
|
442 # Chunkdata: (node, p1, p2, cs, deltabase, delta, flags) |
05131c963767
changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents:
34298
diff
changeset
|
443 yield chunkdata |
05131c963767
changegroup: remove dictionary creation from deltachunk
Durham Goode <durham@fb.com>
parents:
34298
diff
changeset
|
444 chain = chunkdata[0] |
34160
c8b6ed51386b
changegroup: remove changegroup dependency from revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34118
diff
changeset
|
445 |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
446 class cg2unpacker(cg1unpacker): |
26708
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
447 """Unpacker for cg2 streams. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
448 |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
449 cg2 streams add support for generaldelta, so the delta header |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
450 format is slightly different. All other features about the data |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
451 remain the same. |
749d913f24b8
changegroup: document the public surface area of cg?unpackers
Augie Fackler <augie@google.com>
parents:
26707
diff
changeset
|
452 """ |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
453 deltaheader = _CHANGEGROUPV2_DELTA_HEADER |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
454 deltaheadersize = struct.calcsize(deltaheader) |
23896
becfecaf9087
changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents:
23895
diff
changeset
|
455 version = '02' |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
456 |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
457 def _deltaheader(self, headertuple, prevnode): |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
458 node, p1, p2, deltabase, cs = headertuple |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
459 flags = 0 |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
460 return node, p1, p2, deltabase, cs, flags |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
461 |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
462 class cg3unpacker(cg2unpacker): |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
463 """Unpacker for cg3 streams. |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
464 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
465 cg3 streams add support for exchanging treemanifests and revlog |
27753
d4071cc73f46
changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents:
27752
diff
changeset
|
466 flags. It adds the revlog flags to the delta header and an empty chunk |
d4071cc73f46
changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents:
27752
diff
changeset
|
467 separating manifests and files. |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
468 """ |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
469 deltaheader = _CHANGEGROUPV3_DELTA_HEADER |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
470 deltaheadersize = struct.calcsize(deltaheader) |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
471 version = '03' |
27920
da5f23362517
changegroup: cg3 has two empty groups *after* manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27867
diff
changeset
|
472 _grouplistcount = 2 # One list of manifests and one list of files |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
473 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
474 def _deltaheader(self, headertuple, prevnode): |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
475 node, p1, p2, deltabase, cs, flags = headertuple |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
476 return node, p1, p2, deltabase, cs, flags |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
477 |
38352
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
478 def _unpackmanifests(self, repo, revmap, trp, prog): |
83534c4ec58b
changegroup: use progress helper in apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
38197
diff
changeset
|
479 super(cg3unpacker, self)._unpackmanifests(repo, revmap, trp, prog) |
29736
4e7be6e33269
changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents:
29704
diff
changeset
|
480 for chunkdata in iter(self.filelogheader, {}): |
27754
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
481 # If we get here, there are directory manifests in the changegroup |
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
482 d = chunkdata["filename"] |
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
483 repo.ui.debug("adding %s revisions\n" % d) |
30349
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
484 dirlog = repo.manifestlog._revlog.dirlog(d) |
34298
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
485 deltas = self.deltaiter() |
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
486 if not dirlog.addgroup(deltas, revmap, trp): |
27754
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
487 raise error.Abort(_("received dir revlog group is empty")) |
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
488 |
12329
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
489 class headerlessfixup(object): |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
490 def __init__(self, fh, h): |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
491 self._h = h |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
492 self._fh = fh |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
493 def read(self, n): |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
494 if self._h: |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
495 d, self._h = self._h[:n], self._h[n:] |
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
496 if len(d) < n: |
13457
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
497 d += readexactly(self._fh, n - len(d)) |
12329
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
498 return d |
13457
e74fe15dc7fd
changegroup: verify all stream reads
Mads Kiilerich <mads@kiilerich.com>
parents:
13456
diff
changeset
|
499 return readexactly(self._fh, n) |
12329
7458de933f26
bundle: push chunkbuffer down into decompress
Matt Mackall <mpm@selenic.com>
parents:
12044
diff
changeset
|
500 |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
501 @attr.s(slots=True, frozen=True) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
502 class revisiondelta(object): |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
503 """Describes a delta entry in a changegroup. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
504 |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
505 Captured data is sufficient to serialize the delta into multiple |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
506 formats. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
507 """ |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
508 # 20 byte node of this revision. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
509 node = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
510 # 20 byte nodes of parent revisions. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
511 p1node = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
512 p2node = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
513 # 20 byte node of node this delta is against. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
514 basenode = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
515 # 20 byte node of changeset revision this delta is associated with. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
516 linknode = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
517 # 2 bytes of flags to apply to revision data. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
518 flags = attr.ib() |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
519 # Iterable of chunks holding raw delta data. |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
520 deltachunks = attr.ib() |
38922
ee1ea96cf9c9
changegroup: move ellipsisdata() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38874
diff
changeset
|
521 |
22390
e2806b8613ca
changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents:
22070
diff
changeset
|
522 class cg1packer(object): |
e2806b8613ca
changegroup: rename bundle-related functions and classes
Sune Foldager <cryo@cyanite.org>
parents:
22070
diff
changeset
|
523 deltaheader = _CHANGEGROUPV1_DELTA_HEADER |
23896
becfecaf9087
changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents:
23895
diff
changeset
|
524 version = '01' |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
525 def __init__(self, repo, filematcher, bundlecaps=None): |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
526 """Given a source repo, construct a bundler. |
32327
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
527 |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
528 filematcher is a matcher that matches on files to include in the |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
529 changegroup. Used to facilitate sparse changegroups. |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
530 |
32327
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
531 bundlecaps is optional and can be used to specify the set of |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
532 capabilities which can be used to build the bundle. While bundlecaps is |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
533 unused in core Mercurial, extensions rely on this feature to communicate |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
534 capabilities to customize the changegroup packer. |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
535 """ |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
536 assert filematcher |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
537 self._filematcher = filematcher |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
538 |
32327
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
539 # Set of capabilities we can use to build the bundle. |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
540 if bundlecaps is None: |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
541 bundlecaps = set() |
df3cf9422e1b
changegroup: add bundlecaps back
Durham Goode <durham@fb.com>
parents:
32308
diff
changeset
|
542 self._bundlecaps = bundlecaps |
25831
578fc97904da
generaldelta: mark experimental reordering option
Matt Mackall <mpm@selenic.com>
parents:
25823
diff
changeset
|
543 # experimental config: bundle.reorder |
33185
9f95f0bb343b
configitems: register the 'bundle.reorder' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33048
diff
changeset
|
544 reorder = repo.ui.config('bundle', 'reorder') |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
545 if reorder == 'auto': |
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
546 reorder = None |
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
547 else: |
37087
f0b6fbea00cf
stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents:
36843
diff
changeset
|
548 reorder = stringutil.parsebool(reorder) |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
549 self._repo = repo |
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
550 self._reorder = reorder |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
551 if self._repo.ui.verbose and not self._repo.ui.debugflag: |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
552 self._verbosenote = self._repo.ui.note |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
553 else: |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
554 self._verbosenote = lambda s: None |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
555 |
13831
d69c9510d648
changegroup: introduce bundler objects
Matt Mackall <mpm@selenic.com>
parents:
13786
diff
changeset
|
556 def close(self): |
38926
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
557 # Ellipses serving mode. |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
558 getattr(self, 'clrev_to_localrev', {}).clear() |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
559 if getattr(self, 'next_clrev_to_localrev', {}): |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
560 self.clrev_to_localrev = self.next_clrev_to_localrev |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
561 del self.next_clrev_to_localrev |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
562 self.changelog_done = True |
75d6139e69f9
changegroup: move close() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38925
diff
changeset
|
563 |
13831
d69c9510d648
changegroup: introduce bundler objects
Matt Mackall <mpm@selenic.com>
parents:
13786
diff
changeset
|
564 return closechunk() |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
565 |
13831
d69c9510d648
changegroup: introduce bundler objects
Matt Mackall <mpm@selenic.com>
parents:
13786
diff
changeset
|
566 def fileheader(self, fname): |
d69c9510d648
changegroup: introduce bundler objects
Matt Mackall <mpm@selenic.com>
parents:
13786
diff
changeset
|
567 return chunkheader(len(fname)) + fname |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
568 |
29236
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
569 # Extracted both for clarity and for overriding in extensions. |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
570 def _sortgroup(self, store, nodelist, lookup): |
29236
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
571 """Sort nodes for change group and turn them into revnums.""" |
38927
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
572 # Ellipses serving mode. |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
573 # |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
574 # In a perfect world, we'd generate better ellipsis-ified graphs |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
575 # for non-changelog revlogs. In practice, we haven't started doing |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
576 # that yet, so the resulting DAGs for the manifestlog and filelogs |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
577 # are actually full of bogus parentage on all the ellipsis |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
578 # nodes. This has the side effect that, while the contents are |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
579 # correct, the individual DAGs might be completely out of whack in |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
580 # a case like 882681bc3166 and its ancestors (back about 10 |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
581 # revisions or so) in the main hg repo. |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
582 # |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
583 # The one invariant we *know* holds is that the new (potentially |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
584 # bogus) DAG shape will be valid if we order the nodes in the |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
585 # order that they're introduced in dramatis personae by the |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
586 # changelog, so what we do is we sort the non-changelog histories |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
587 # by the order in which they are used by the changelog. |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
588 if util.safehasattr(self, 'full_nodes') and self.clnode_to_rev: |
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
589 key = lambda n: self.clnode_to_rev[lookup(n)] |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
590 return [store.rev(n) for n in sorted(nodelist, key=key)] |
38927
c9315bc578bc
changegroup: move _sortgroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38926
diff
changeset
|
591 |
29236
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
592 # for generaldelta revlogs, we linearize the revs; this will both be |
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
593 # much quicker and generate a much smaller bundle |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
594 if (store._generaldelta and self._reorder is None) or self._reorder: |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
595 dag = dagutil.revlogdag(store) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
596 return dag.linearize(set(store.rev(n) for n in nodelist)) |
29236
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
597 else: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
598 return sorted([store.rev(n) for n in nodelist]) |
29236
1b7d907ec18a
changegroup: extract method that sorts nodes to send
Augie Fackler <augie@google.com>
parents:
28666
diff
changeset
|
599 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
600 def group(self, nodelist, store, lookup, units=None): |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
601 """Calculate a delta group, yielding a sequence of changegroup chunks |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
602 (strings). |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
603 |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
604 Given a list of changeset revs, return a set of deltas and |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
605 metadata corresponding to nodes. The first delta is |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
606 first parent(nodelist[0]) -> nodelist[0], the receiver is |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
607 guaranteed to have this parent as it has all history before |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
608 these changesets. In the case firstparent is nullrev the |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
609 changegroup starts with a full revision. |
19208
0b564cf359a7
bundle-ng: move progress handling out of the linkrev callback
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19207
diff
changeset
|
610 |
0b564cf359a7
bundle-ng: move progress handling out of the linkrev callback
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19207
diff
changeset
|
611 If units is not None, progress detail will be generated, units specifies |
0b564cf359a7
bundle-ng: move progress handling out of the linkrev callback
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19207
diff
changeset
|
612 the type of revlog that is touched (changelog, manifest, etc.). |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
613 """ |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
614 # if we don't have any revisions touched by these changesets, bail |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
615 if len(nodelist) == 0: |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
616 yield self.close() |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
617 return |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
618 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
619 revs = self._sortgroup(store, nodelist, lookup) |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
620 |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
621 # add the parent of the first rev |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
622 p = store.parentrevs(revs[0])[0] |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
623 revs.insert(0, p) |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
624 |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
625 # build deltas |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
626 progress = None |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
627 if units is not None: |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
628 progress = self._repo.ui.makeprogress(_('bundling'), unit=units, |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
629 total=(len(revs) - 1)) |
38823
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38557
diff
changeset
|
630 for r in pycompat.xrange(len(revs) - 1): |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
631 if progress: |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
632 progress.update(r + 1) |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
633 prev, curr = revs[r], revs[r + 1] |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
634 linknode = lookup(store.node(curr)) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
635 for c in self.revchunk(store, curr, prev, linknode): |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
636 yield c |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
637 |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
638 if progress: |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
639 progress.complete() |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
640 yield self.close() |
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
641 |
19289
6ea1f858efd9
bundle: refactor changegroup prune to be its own function
Durham Goode <durham@fb.com>
parents:
19208
diff
changeset
|
642 # filter any nodes that claim to be part of the known set |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
643 def prune(self, store, missing, commonrevs): |
38846
5742d0428ed9
changegroup: inline prune() logic from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38834
diff
changeset
|
644 # TODO this violates storage abstraction for manifests. |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
645 if isinstance(store, manifest.manifestrevlog): |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
646 if not self._filematcher.visitdir(store._dir[:-1] or '.'): |
38846
5742d0428ed9
changegroup: inline prune() logic from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38834
diff
changeset
|
647 return [] |
5742d0428ed9
changegroup: inline prune() logic from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38834
diff
changeset
|
648 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
649 rr, rl = store.rev, store.linkrev |
19289
6ea1f858efd9
bundle: refactor changegroup prune to be its own function
Durham Goode <durham@fb.com>
parents:
19208
diff
changeset
|
650 return [n for n in missing if rl(rr(n)) not in commonrevs] |
6ea1f858efd9
bundle: refactor changegroup prune to be its own function
Durham Goode <durham@fb.com>
parents:
19208
diff
changeset
|
651 |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
652 def _packmanifests(self, dir, mfnodes, lookuplinknode): |
26711
0ef0aec56090
changegroup: move manifest packing into a separate function
Augie Fackler <augie@google.com>
parents:
26710
diff
changeset
|
653 """Pack flat manifests into a changegroup stream.""" |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
654 assert not dir |
30349
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
655 for chunk in self.group(mfnodes, self._repo.manifestlog._revlog, |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
656 lookuplinknode, units=_('manifests')): |
26711
0ef0aec56090
changegroup: move manifest packing into a separate function
Augie Fackler <augie@google.com>
parents:
26710
diff
changeset
|
657 yield chunk |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
658 |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
659 def _manifestsdone(self): |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
660 return '' |
26711
0ef0aec56090
changegroup: move manifest packing into a separate function
Augie Fackler <augie@google.com>
parents:
26710
diff
changeset
|
661 |
19204
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
662 def generate(self, commonrevs, clnodes, fastpathlinkrev, source): |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
663 '''yield a sequence of changegroup chunks (strings)''' |
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
664 repo = self._repo |
24978
f52560c64953
changegroup: drop _changelog and _manifest properties
Martin von Zweigbergk <martinvonz@google.com>
parents:
24977
diff
changeset
|
665 cl = repo.changelog |
19204
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
666 |
23381
cc0ff93d0c0c
changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents:
23226
diff
changeset
|
667 clrevorder = {} |
19204
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
668 mfs = {} # needed manifests |
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
669 fnodes = {} # needed file nodes |
38929
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
670 mfl = repo.manifestlog |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
671 # TODO violates storage abstraction. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
672 mfrevlog = mfl._revlog |
28241
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
673 changedfiles = set() |
19204
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
674 |
38929
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
675 ellipsesmode = util.safehasattr(self, 'full_nodes') |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
676 |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
677 # Callback for the changelog, used to collect changed files and |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
678 # manifest nodes. |
19207
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
679 # Returns the linkrev node (identity in the changelog case). |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
680 def lookupcl(x): |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
681 c = cl.read(x) |
23381
cc0ff93d0c0c
changegroup: fix file linkrevs during reorders (issue4462)
Durham Goode <durham@fb.com>
parents:
23226
diff
changeset
|
682 clrevorder[x] = len(clrevorder) |
38929
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
683 |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
684 if ellipsesmode: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
685 # Only update mfs if x is going to be sent. Otherwise we |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
686 # end up with bogus linkrevs specified for manifests and |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
687 # we skip some manifest nodes that we should otherwise |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
688 # have sent. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
689 if (x in self.full_nodes |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
690 or cl.rev(x) in self.precomputed_ellipsis): |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
691 n = c[0] |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
692 # Record the first changeset introducing this manifest |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
693 # version. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
694 mfs.setdefault(n, x) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
695 # Set this narrow-specific dict so we have the lowest |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
696 # manifest revnum to look up for this cl revnum. (Part of |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
697 # mapping changelog ellipsis parents to manifest ellipsis |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
698 # parents) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
699 self.next_clrev_to_localrev.setdefault(cl.rev(x), |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
700 mfrevlog.rev(n)) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
701 # We can't trust the changed files list in the changeset if the |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
702 # client requested a shallow clone. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
703 if self.is_shallow: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
704 changedfiles.update(mfl[c[0]].read().keys()) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
705 else: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
706 changedfiles.update(c[3]) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
707 else: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
708 |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
709 n = c[0] |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
710 # record the first changeset introducing this manifest version |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
711 mfs.setdefault(n, x) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
712 # Record a complete list of potentially-changed files in |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
713 # this manifest. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
714 changedfiles.update(c[3]) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
715 |
19207
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
716 return x |
19204
e9c5b1c246dc
bundle-ng: move bundle generation to changegroup.py
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19202
diff
changeset
|
717 |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
718 self._verbosenote(_('uncompressed size of bundle content:\n')) |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
719 size = 0 |
24912
e285b98c65cc
changegroup.group: drop 'reorder' parameter
Martin von Zweigbergk <martinvonz@google.com>
parents:
24911
diff
changeset
|
720 for chunk in self.group(clnodes, cl, lookupcl, units=_('changesets')): |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
721 size += len(chunk) |
23224
f4ab47ccefde
changegroup: don't define lookupmf() until it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents:
22971
diff
changeset
|
722 yield chunk |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
723 self._verbosenote(_('%8.i (changelog)\n') % size) |
23224
f4ab47ccefde
changegroup: don't define lookupmf() until it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents:
22971
diff
changeset
|
724 |
24977
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
725 # We need to make sure that the linkrev in the changegroup refers to |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
726 # the first changeset that introduced the manifest or file revision. |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
727 # The fastpath is usually safer than the slowpath, because the filelogs |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
728 # are walked in revlog order. |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
729 # |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
730 # When taking the slowpath with reorder=None and the manifest revlog |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
731 # uses generaldelta, the manifest may be walked in the "wrong" order. |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
732 # Without 'clrevorder', we would get an incorrect linkrev (see fix in |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
733 # cc0ff93d0c0c). |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
734 # |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
735 # When taking the fastpath, we are only vulnerable to reordering |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
736 # of the changelog itself. The changelog never uses generaldelta, so |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
737 # it is only reordered when reorder=True. To handle this case, we |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
738 # simply take the slowpath, which already has the 'clrevorder' logic. |
4289383cb9d2
changegroup: document the cases where reordering complicates linkrevs
Martin von Zweigbergk <martinvonz@google.com>
parents:
24976
diff
changeset
|
739 # This was also fixed in cc0ff93d0c0c. |
24976
147d8892fc4b
changegroup: extract condition for linkrev fastpath
Martin von Zweigbergk <martinvonz@google.com>
parents:
24912
diff
changeset
|
740 fastpathlinkrev = fastpathlinkrev and not self._reorder |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
741 # Treemanifests don't work correctly with fastpathlinkrev |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
742 # either, because we don't discover which directory nodes to |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
743 # send along with files. This could probably be fixed. |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
744 fastpathlinkrev = fastpathlinkrev and ( |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
745 'treemanifest' not in repo.requirements) |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
746 |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
747 for chunk in self.generatemanifests(commonrevs, clrevorder, |
34161
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
748 fastpathlinkrev, mfs, fnodes, source): |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
749 yield chunk |
38929
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
750 |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
751 if ellipsesmode: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
752 mfdicts = None |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
753 if self.is_shallow: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
754 mfdicts = [(self._repo.manifestlog[n].read(), lr) |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
755 for (n, lr) in mfs.iteritems()] |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
756 |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
757 mfs.clear() |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
758 clrevs = set(cl.rev(x) for x in clnodes) |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
759 |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
760 if not fastpathlinkrev: |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
761 def linknodes(unused, fname): |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
762 return fnodes.get(fname, {}) |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
763 else: |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
764 cln = cl.node |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
765 def linknodes(filerevlog, fname): |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
766 llr = filerevlog.linkrev |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
767 fln = filerevlog.node |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
768 revs = ((r, llr(r)) for r in filerevlog) |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
769 return dict((fln(r), cln(lr)) for r, lr in revs if lr in clrevs) |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
770 |
38929
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
771 if ellipsesmode: |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
772 # We need to pass the mfdicts variable down into |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
773 # generatefiles(), but more than one command might have |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
774 # wrapped generatefiles so we can't modify the function |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
775 # signature. Instead, we pass the data to ourselves using an |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
776 # instance attribute. I'm sorry. |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
777 self._mfdicts = mfdicts |
d706c77449f9
changegroup: move generate() modifications from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38928
diff
changeset
|
778 |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
779 for chunk in self.generatefiles(changedfiles, linknodes, commonrevs, |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
780 source): |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
781 yield chunk |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
782 |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
783 yield self.close() |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
784 |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
785 if clnodes: |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
786 repo.hook('outgoing', node=hex(clnodes[0]), source=source) |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
787 |
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
788 def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, mfs, |
34161
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
789 fnodes, source): |
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
790 """Returns an iterator of changegroup chunks containing manifests. |
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
791 |
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
792 `source` is unused here, but is used by extensions like remotefilelog to |
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
793 change what is sent based in pulls vs pushes, etc. |
75cc1f1e11f2
changegroup: add source parameter to generatemanifests
Durham Goode <durham@fb.com>
parents:
34160
diff
changeset
|
794 """ |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
795 repo = self._repo |
30308
bce79dfcf5e4
manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents:
30282
diff
changeset
|
796 mfl = repo.manifestlog |
bce79dfcf5e4
manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents:
30282
diff
changeset
|
797 dirlog = mfl._revlog.dirlog |
28232
829d369fc5a8
changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents:
28231
diff
changeset
|
798 tmfnodes = {'': mfs} |
28227
1c36cc8e7870
changegroup: extract generatemanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27953
diff
changeset
|
799 |
19207
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
800 # Callback for the manifest, used to collect linkrevs for filelog |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
801 # revisions. |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
802 # Returns the linkrev node (collected in lookupcl). |
35051
d80380ba8e7d
changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents:
34734
diff
changeset
|
803 def makelookupmflinknode(dir, nodes): |
28231
3faba927dd93
changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents:
28230
diff
changeset
|
804 if fastpathlinkrev: |
3faba927dd93
changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents:
28230
diff
changeset
|
805 assert not dir |
3faba927dd93
changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents:
28230
diff
changeset
|
806 return mfs.__getitem__ |
3faba927dd93
changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents:
28230
diff
changeset
|
807 |
27239
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
808 def lookupmflinknode(x): |
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
809 """Callback for looking up the linknode for manifests. |
27219
beb60a898dd0
changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents:
27218
diff
changeset
|
810 |
27239
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
811 Returns the linkrev node for the specified manifest. |
27219
beb60a898dd0
changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents:
27218
diff
changeset
|
812 |
27239
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
813 SIDE EFFECT: |
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
814 |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
815 1) fclnodes gets populated with the list of relevant |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
816 file nodes if we're not using fastpathlinkrev |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
817 2) When treemanifests are in use, collects treemanifest nodes |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
818 to send |
27219
beb60a898dd0
changegroup: document manifest linkrev callback some more
Augie Fackler <augie@google.com>
parents:
27218
diff
changeset
|
819 |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
820 Note that this means manifests must be completely sent to |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
821 the client before you can trust the list of files and |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
822 treemanifests to send. |
27239
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
823 """ |
35051
d80380ba8e7d
changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents:
34734
diff
changeset
|
824 clnode = nodes[x] |
30308
bce79dfcf5e4
manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com>
parents:
30282
diff
changeset
|
825 mdata = mfl.get(dir, x).readfast(shallow=True) |
28241
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
826 for p, n, fl in mdata.iterentries(): |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
827 if fl == 't': # subdirectory manifest |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
828 subdir = dir + p + '/' |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
829 tmfclnodes = tmfnodes.setdefault(subdir, {}) |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
830 tmfclnode = tmfclnodes.setdefault(n, clnode) |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
831 if clrevorder[clnode] < clrevorder[tmfclnode]: |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
832 tmfclnodes[n] = clnode |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
833 else: |
a4286175ecba
changegroup: drop special-casing of flat manifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
28240
diff
changeset
|
834 f = dir + p |
28240
1ac8ce137377
changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents:
28232
diff
changeset
|
835 fclnodes = fnodes.setdefault(f, {}) |
1ac8ce137377
changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents:
28232
diff
changeset
|
836 fclnode = fclnodes.setdefault(n, clnode) |
1ac8ce137377
changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents:
28232
diff
changeset
|
837 if clrevorder[clnode] < clrevorder[fclnode]: |
1ac8ce137377
changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents:
28232
diff
changeset
|
838 fclnodes[n] = clnode |
27239
65c47779bcb5
changegroup: remove one special case from lookupmflinknode
Augie Fackler <augie@google.com>
parents:
27238
diff
changeset
|
839 return clnode |
28231
3faba927dd93
changegroup: introduce makelookupmflinknode(dir)
Martin von Zweigbergk <martinvonz@google.com>
parents:
28230
diff
changeset
|
840 return lookupmflinknode |
19206
6308896b1d4a
bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents:
19204
diff
changeset
|
841 |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
842 size = 0 |
28232
829d369fc5a8
changegroup: write root manifests and subdir manifests in a single loop
Martin von Zweigbergk <martinvonz@google.com>
parents:
28231
diff
changeset
|
843 while tmfnodes: |
35051
d80380ba8e7d
changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents:
34734
diff
changeset
|
844 dir, nodes = tmfnodes.popitem() |
28240
1ac8ce137377
changegroup: fix treemanifests on merges
Martin von Zweigbergk <martinvonz@google.com>
parents:
28232
diff
changeset
|
845 prunednodes = self.prune(dirlog(dir), nodes, commonrevs) |
29371
1b699c7eb2b7
changegroup: don't send empty subdirectory manifest groups
Martin von Zweigbergk <martinvonz@google.com>
parents:
29236
diff
changeset
|
846 if not dir or prunednodes: |
1b699c7eb2b7
changegroup: don't send empty subdirectory manifest groups
Martin von Zweigbergk <martinvonz@google.com>
parents:
29236
diff
changeset
|
847 for x in self._packmanifests(dir, prunednodes, |
35051
d80380ba8e7d
changegroup: use any node, not min(), in treemanifest's generatemanifests
Kyle Lippincott <spectral@google.com>
parents:
34734
diff
changeset
|
848 makelookupmflinknode(dir, nodes)): |
29371
1b699c7eb2b7
changegroup: don't send empty subdirectory manifest groups
Martin von Zweigbergk <martinvonz@google.com>
parents:
29236
diff
changeset
|
849 size += len(x) |
1b699c7eb2b7
changegroup: don't send empty subdirectory manifest groups
Martin von Zweigbergk <martinvonz@google.com>
parents:
29236
diff
changeset
|
850 yield x |
28229
8e13b2379407
changegroup: include subdirectory manifests in verbose size
Martin von Zweigbergk <martinvonz@google.com>
parents:
28228
diff
changeset
|
851 self._verbosenote(_('%8.i (manifests)\n') % size) |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
852 yield self._manifestsdone() |
19206
6308896b1d4a
bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents:
19204
diff
changeset
|
853 |
24897
5c35a6040352
changegroup: document that 'source' parameter exists for extensions
Martin von Zweigbergk <martinvonz@google.com>
parents:
24896
diff
changeset
|
854 # The 'source' parameter is useful for extensions |
19334
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
855 def generatefiles(self, changedfiles, linknodes, commonrevs, source): |
38928
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
856 changedfiles = list(filter(self._filematcher, changedfiles)) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
857 |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
858 if getattr(self, 'is_shallow', False): |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
859 # See comment in generate() for why this sadness is a thing. |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
860 mfdicts = self._mfdicts |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
861 del self._mfdicts |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
862 # In a shallow clone, the linknodes callback needs to also include |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
863 # those file nodes that are in the manifests we sent but weren't |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
864 # introduced by those manifests. |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
865 commonctxs = [self._repo[c] for c in commonrevs] |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
866 oldlinknodes = linknodes |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
867 clrev = self._repo.changelog.rev |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
868 |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
869 # Defining this function has a side-effect of overriding the |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
870 # function of the same name that was passed in as an argument. |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
871 # TODO have caller pass in appropriate function. |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
872 def linknodes(flog, fname): |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
873 for c in commonctxs: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
874 try: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
875 fnode = c.filenode(fname) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
876 self.clrev_to_localrev[c.rev()] = flog.rev(fnode) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
877 except error.ManifestLookupError: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
878 pass |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
879 links = oldlinknodes(flog, fname) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
880 if len(links) != len(mfdicts): |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
881 for mf, lr in mfdicts: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
882 fnode = mf.get(fname, None) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
883 if fnode in links: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
884 links[fnode] = min(links[fnode], lr, key=clrev) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
885 elif fnode: |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
886 links[fnode] = lr |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
887 return links |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
888 |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
889 return self._generatefiles(changedfiles, linknodes, commonrevs, source) |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
890 |
a06aab274aef
changegroup: move generatefiles() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38927
diff
changeset
|
891 def _generatefiles(self, changedfiles, linknodes, commonrevs, source): |
19334
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
892 repo = self._repo |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
893 progress = repo.ui.makeprogress(_('bundling'), unit=_('files'), |
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
894 total=len(changedfiles)) |
19334
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
895 for i, fname in enumerate(sorted(changedfiles)): |
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
896 filerevlog = repo.file(fname) |
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
897 if not filerevlog: |
37339
5859800edfc5
changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37135
diff
changeset
|
898 raise error.Abort(_("empty or missing file data for %s") % |
5859800edfc5
changegroup: remove "revlog" from error message
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37135
diff
changeset
|
899 fname) |
19334
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
900 |
95a49112e7ab
bundle: move file chunk generation to it's own function
Durham Goode <durham@fb.com>
parents:
19325
diff
changeset
|
901 linkrevnodes = linknodes(filerevlog, fname) |
19207
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
902 # Lookup for filenodes, we collected the linkrev nodes above in the |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
903 # fastpath case and with lookupmf in the slowpath case. |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
904 def lookupfilelog(x): |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
905 return linkrevnodes[x] |
a67e1380dfbd
bundle-ng: simplify lookup and state handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
19206
diff
changeset
|
906 |
24896
9183cb6886ef
changegroup: removed unused 'source' parameter from prune()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24717
diff
changeset
|
907 filenodes = self.prune(filerevlog, linkrevnodes, commonrevs) |
19206
6308896b1d4a
bundle-ng: simplify bundle10.generate
Sune Foldager <cryo@cyanite.org>
parents:
19204
diff
changeset
|
908 if filenodes: |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
909 progress.update(i + 1, item=fname) |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
910 h = self.fileheader(fname) |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
911 size = len(h) |
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
912 yield h |
24912
e285b98c65cc
changegroup.group: drop 'reorder' parameter
Martin von Zweigbergk <martinvonz@google.com>
parents:
24911
diff
changeset
|
913 for chunk in self.group(filenodes, filerevlog, lookupfilelog): |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
914 size += len(chunk) |
19202
0455fc94ae00
bundle-ng: move gengroup into bundler, pass repo object to bundler
Sune Foldager <cryo@cyanite.org>
parents:
19201
diff
changeset
|
915 yield chunk |
23748
4ab66de46a96
bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents:
23382
diff
changeset
|
916 self._verbosenote(_('%8.i %s\n') % (size, fname)) |
38416
1c5c4a5dd86d
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38388
diff
changeset
|
917 progress.complete() |
19200
4cfdec944edf
bundle-ng: move group into the bundler
Sune Foldager <cryo@cyanite.org>
parents:
19199
diff
changeset
|
918 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
919 def deltaparent(self, store, rev, p1, p2, prev): |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
920 if not store.candelta(prev, rev): |
35839
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
921 raise error.ProgrammingError('cg1 should not be used in this case') |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
922 return prev |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
923 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
924 def revchunk(self, store, rev, prev, linknode): |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
925 if util.safehasattr(self, 'full_nodes'): |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
926 fn = self._revisiondeltanarrow |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
927 else: |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
928 fn = self._revisiondeltanormal |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
929 |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
930 delta = fn(store, rev, prev, linknode) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
931 if not delta: |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
932 return |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
933 |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
934 meta = self.builddeltaheader(delta.node, delta.p1node, delta.p2node, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
935 delta.basenode, delta.linknode, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
936 delta.flags) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
937 l = len(meta) + sum(len(x) for x in delta.deltachunks) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
938 |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
939 yield chunkheader(l) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
940 yield meta |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
941 for x in delta.deltachunks: |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
942 yield x |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
943 |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
944 def _revisiondeltanormal(self, store, rev, prev, linknode): |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
945 node = store.node(rev) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
946 p1, p2 = store.parentrevs(rev) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
947 base = self.deltaparent(store, rev, p1, p2, prev) |
14143
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
948 |
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
949 prefix = '' |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
950 if store.iscensored(base) or store.iscensored(rev): |
24190
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
951 try: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
952 delta = store.revision(node, raw=True) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25624
diff
changeset
|
953 except error.CensoredNodeError as e: |
24190
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
954 delta = e.tombstone |
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
955 if base == nullrev: |
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
956 prefix = mdiff.trivialdiffheader(len(delta)) |
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
957 else: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
958 baselen = store.rawsize(base) |
24190
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
959 prefix = mdiff.replacediffheader(baselen, len(delta)) |
903c7e8c97ad
changegroup: emit full-replacement deltas if either revision is censored
Mike Edgar <adgar@google.com>
parents:
24180
diff
changeset
|
960 elif base == nullrev: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
961 delta = store.revision(node, raw=True) |
14143
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
962 prefix = mdiff.trivialdiffheader(len(delta)) |
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
963 else: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
964 delta = store.revdiff(base, rev) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
965 p1n, p2n = store.parents(node) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
966 |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
967 return revisiondelta( |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
968 node=node, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
969 p1node=p1n, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
970 p2node=p2n, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
971 basenode=store.node(base), |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
972 linknode=linknode, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
973 flags=store.flags(rev), |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
974 deltachunks=(prefix, delta), |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
975 ) |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
976 |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
977 def _revisiondeltanarrow(self, store, rev, prev, linknode): |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
978 # build up some mapping information that's useful later. See |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
979 # the local() nested function below. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
980 if not self.changelog_done: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
981 self.clnode_to_rev[linknode] = rev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
982 linkrev = rev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
983 self.clrev_to_localrev[linkrev] = rev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
984 else: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
985 linkrev = self.clnode_to_rev[linknode] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
986 self.clrev_to_localrev[linkrev] = rev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
987 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
988 # This is a node to send in full, because the changeset it |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
989 # corresponds to was a full changeset. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
990 if linknode in self.full_nodes: |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
991 return self._revisiondeltanormal(store, rev, prev, linknode) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
992 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
993 # At this point, a node can either be one we should skip or an |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
994 # ellipsis. If it's not an ellipsis, bail immediately. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
995 if linkrev not in self.precomputed_ellipsis: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
996 return |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
997 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
998 linkparents = self.precomputed_ellipsis[linkrev] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
999 def local(clrev): |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1000 """Turn a changelog revnum into a local revnum. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1001 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1002 The ellipsis dag is stored as revnums on the changelog, |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1003 but when we're producing ellipsis entries for |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1004 non-changelog revlogs, we need to turn those numbers into |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1005 something local. This does that for us, and during the |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1006 changelog sending phase will also expand the stored |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1007 mappings as needed. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1008 """ |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1009 if clrev == nullrev: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1010 return nullrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1011 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1012 if not self.changelog_done: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1013 # If we're doing the changelog, it's possible that we |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1014 # have a parent that is already on the client, and we |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1015 # need to store some extra mapping information so that |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1016 # our contained ellipsis nodes will be able to resolve |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1017 # their parents. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1018 if clrev not in self.clrev_to_localrev: |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1019 clnode = store.node(clrev) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1020 self.clnode_to_rev[clnode] = clrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1021 return clrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1022 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1023 # Walk the ellipsis-ized changelog breadth-first looking for a |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1024 # change that has been linked from the current revlog. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1025 # |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1026 # For a flat manifest revlog only a single step should be necessary |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1027 # as all relevant changelog entries are relevant to the flat |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1028 # manifest. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1029 # |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1030 # For a filelog or tree manifest dirlog however not every changelog |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1031 # entry will have been relevant, so we need to skip some changelog |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1032 # nodes even after ellipsis-izing. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1033 walk = [clrev] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1034 while walk: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1035 p = walk[0] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1036 walk = walk[1:] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1037 if p in self.clrev_to_localrev: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1038 return self.clrev_to_localrev[p] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1039 elif p in self.full_nodes: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1040 walk.extend([pp for pp in self._repo.changelog.parentrevs(p) |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1041 if pp != nullrev]) |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1042 elif p in self.precomputed_ellipsis: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1043 walk.extend([pp for pp in self.precomputed_ellipsis[p] |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1044 if pp != nullrev]) |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1045 else: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1046 # In this case, we've got an ellipsis with parents |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1047 # outside the current bundle (likely an |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1048 # incremental pull). We "know" that we can use the |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1049 # value of this same revlog at whatever revision |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1050 # is pointed to by linknode. "Know" is in scare |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1051 # quotes because I haven't done enough examination |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1052 # of edge cases to convince myself this is really |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1053 # a fact - it works for all the (admittedly |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1054 # thorough) cases in our testsuite, but I would be |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1055 # somewhat unsurprised to find a case in the wild |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1056 # where this breaks down a bit. That said, I don't |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1057 # know if it would hurt anything. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1058 for i in pycompat.xrange(rev, 0, -1): |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1059 if store.linkrev(i) == clrev: |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1060 return i |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1061 # We failed to resolve a parent for this node, so |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1062 # we crash the changegroup construction. |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1063 raise error.Abort( |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1064 'unable to resolve parent while packing %r %r' |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1065 ' for changeset %r' % (store.indexfile, rev, clrev)) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1066 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1067 return nullrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1068 |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1069 if not linkparents or ( |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1070 store.parentrevs(rev) == (nullrev, nullrev)): |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1071 p1, p2 = nullrev, nullrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1072 elif len(linkparents) == 1: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1073 p1, = sorted(local(p) for p in linkparents) |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1074 p2 = nullrev |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1075 else: |
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1076 p1, p2 = sorted(local(p) for p in linkparents) |
38931
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1077 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1078 n = store.node(rev) |
38931
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1079 p1n, p2n = store.node(p1), store.node(p2) |
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1080 flags = store.flags(rev) |
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1081 flags |= revlog.REVIDX_ELLIPSIS |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1082 |
38931
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1083 # TODO: try and actually send deltas for ellipsis data blocks |
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1084 data = store.revision(n) |
eb022ce9e505
changegroup: inline ellipsisdata()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38930
diff
changeset
|
1085 diffheader = mdiff.trivialdiffheader(len(data)) |
38932
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1086 |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1087 return revisiondelta( |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1088 node=n, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1089 p1node=p1n, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1090 p2node=p2n, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1091 basenode=nullid, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1092 linknode=linknode, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1093 flags=flags, |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1094 deltachunks=(diffheader, data), |
23d582caae30
changegroup: capture revision delta in a data structure
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38931
diff
changeset
|
1095 ) |
38925
66cf046ef60f
changegroup: move revchunk() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38924
diff
changeset
|
1096 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1097 def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags): |
14143
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
1098 # do nothing with basenode, it is implicitly the previous one in HG10 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1099 # do nothing with flags, it is implicitly 0 for cg1 and cg2 |
14143
da635d3c5620
changegroup: new bundler API
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
14141
diff
changeset
|
1100 return struct.pack(self.deltaheader, node, p1n, p2n, linknode) |
20925
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1101 |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1102 class cg2packer(cg1packer): |
23896
becfecaf9087
changegroup.writebundle: HG2Y support
Eric Sumner <ericsumner@fb.com>
parents:
23895
diff
changeset
|
1103 version = '02' |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1104 deltaheader = _CHANGEGROUPV2_DELTA_HEADER |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1105 |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1106 def __init__(self, repo, filematcher, bundlecaps=None): |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1107 super(cg2packer, self).__init__(repo, filematcher, |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1108 bundlecaps=bundlecaps) |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1109 |
24911
5447b8523fef
cg2packer: set reorder=False in __init__ instead of in group()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24910
diff
changeset
|
1110 if self._reorder is None: |
5447b8523fef
cg2packer: set reorder=False in __init__ instead of in group()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24910
diff
changeset
|
1111 # Since generaldelta is directly supported by cg2, reordering |
5447b8523fef
cg2packer: set reorder=False in __init__ instead of in group()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24910
diff
changeset
|
1112 # generally doesn't help, so we disable it by default (treating |
5447b8523fef
cg2packer: set reorder=False in __init__ instead of in group()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24910
diff
changeset
|
1113 # bundle.reorder=auto just like bundle.reorder=False). |
5447b8523fef
cg2packer: set reorder=False in __init__ instead of in group()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24910
diff
changeset
|
1114 self._reorder = False |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1115 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1116 def deltaparent(self, store, rev, p1, p2, prev): |
38924
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1117 # Narrow ellipses mode. |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1118 if util.safehasattr(self, 'full_nodes'): |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1119 # TODO: send better deltas when in narrow mode. |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1120 # |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1121 # changegroup.group() loops over revisions to send, |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1122 # including revisions we'll skip. What this means is that |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1123 # `prev` will be a potentially useless delta base for all |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1124 # ellipsis nodes, as the client likely won't have it. In |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1125 # the future we should do bookkeeping about which nodes |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1126 # have been sent to the client, and try to be |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1127 # significantly smarter about delta bases. This is |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1128 # slightly tricky because this same code has to work for |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1129 # all revlogs, and we don't have the linkrev/linknode here. |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1130 return p1 |
5839a170357d
changegroup: move deltaparent() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38923
diff
changeset
|
1131 |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1132 dp = store.deltaparent(rev) |
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1133 if dp == nullrev and store.storedeltachains: |
30211
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1134 # Avoid sending full revisions when delta parent is null. Pick prev |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1135 # in that case. It's tempting to pick p1 in this case, as p1 will |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1136 # be smaller in the common case. However, computing a delta against |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1137 # p1 may require resolving the raw text of p1, which could be |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1138 # expensive. The revlog caches should have prev cached, meaning |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1139 # less CPU for changegroup generation. There is likely room to add |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1140 # a flag and/or config option to control this behavior. |
35839
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
1141 base = prev |
30211
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1142 elif dp == nullrev: |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1143 # revlog is configured to use full snapshot for a reason, |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1144 # stick to full snapshot. |
35839
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
1145 base = nullrev |
30211
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1146 elif dp not in (p1, p2, prev): |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1147 # Pick prev when we can't be sure remote has the base revision. |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1148 return prev |
6b0741d6d234
changegroup: skip delta when the underlying revlog do not use them
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30153
diff
changeset
|
1149 else: |
35839
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
1150 base = dp |
38930
205c98e2f1ba
changegroup: rename "revlog" variables
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38929
diff
changeset
|
1151 if base != nullrev and not store.candelta(base, rev): |
35839
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
1152 base = nullrev |
d031609b3cb7
changegroup: do not delta lfs revisions
Jun Wu <quark@fb.com>
parents:
35754
diff
changeset
|
1153 return base |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1154 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1155 def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags): |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1156 # Do nothing with flags, it is implicitly 0 in cg1 and cg2 |
23181
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1157 return struct.pack(self.deltaheader, node, p1n, p2n, basenode, linknode) |
832b7ef275c8
changegroup: introduce cg2packer/unpacker
Sune Foldager <cryo@cyanite.org>
parents:
23178
diff
changeset
|
1158 |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1159 class cg3packer(cg2packer): |
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1160 version = '03' |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1161 deltaheader = _CHANGEGROUPV3_DELTA_HEADER |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1162 |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1163 def _packmanifests(self, dir, mfnodes, lookuplinknode): |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1164 if dir: |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1165 yield self.fileheader(dir) |
30349
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
1166 |
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
1167 dirlog = self._repo.manifestlog._revlog.dirlog(dir) |
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
1168 for chunk in self.group(mfnodes, dirlog, lookuplinknode, |
6cdfb7e15a35
changegroup: remove remaining uses of repo.manifest
Durham Goode <durham@fb.com>
parents:
30308
diff
changeset
|
1169 units=_('manifests')): |
28228
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1170 yield chunk |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1171 |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1172 def _manifestsdone(self): |
abf120262683
changegroup: make _packmanifests() dumber
Martin von Zweigbergk <martinvonz@google.com>
parents:
28227
diff
changeset
|
1173 return self.close() |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1174 |
27433
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1175 def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags): |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1176 return struct.pack( |
12f727a5b434
changegroup: add flags field to cg3 delta header
Mike Edgar <adgar@google.com>
parents:
27432
diff
changeset
|
1177 self.deltaheader, node, p1n, p2n, basenode, linknode, flags) |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1178 |
27751
a40e2f7fe49d
changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents:
27739
diff
changeset
|
1179 _packermap = {'01': (cg1packer, cg1unpacker), |
26709
42733e956887
changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents:
26708
diff
changeset
|
1180 # cg2 adds support for exchanging generaldelta |
42733e956887
changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents:
26708
diff
changeset
|
1181 '02': (cg2packer, cg2unpacker), |
27753
d4071cc73f46
changegroup3: add empty chunk separating directories and files
Martin von Zweigbergk <martinvonz@google.com>
parents:
27752
diff
changeset
|
1182 # cg3 adds support for exchanging revlog flags and treemanifests |
27432
77d25b913f80
changegroup: introduce cg3, which has support for exchanging treemanifests
Augie Fackler <augie@google.com>
parents:
27241
diff
changeset
|
1183 '03': (cg3packer, cg3unpacker), |
26709
42733e956887
changegroup: reformat packermap and add comment
Augie Fackler <augie@google.com>
parents:
26708
diff
changeset
|
1184 } |
23168
a92ba36a1a9d
changegroup: add a "packermap" dictionary to track different packer versions
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22971
diff
changeset
|
1185 |
30632
7ace5304fec5
changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30631
diff
changeset
|
1186 def allsupportedversions(repo): |
27928
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1187 versions = set(_packermap.keys()) |
30632
7ace5304fec5
changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30631
diff
changeset
|
1188 if not (repo.ui.configbool('experimental', 'changegroup3') or |
30633
a001cd7296a5
changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30632
diff
changeset
|
1189 repo.ui.configbool('experimental', 'treemanifest') or |
a001cd7296a5
changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30632
diff
changeset
|
1190 'treemanifest' in repo.requirements): |
30631
438532c99b54
changegroup: simplify 'allsupportedversions' logic
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30594
diff
changeset
|
1191 versions.discard('03') |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1192 return versions |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1193 |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1194 # Changegroup versions that can be applied to the repo |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1195 def supportedincomingversions(repo): |
30633
a001cd7296a5
changegroup: simplify logic around enabling changegroup 03
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30632
diff
changeset
|
1196 return allsupportedversions(repo) |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1197 |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1198 # Changegroup versions that can be created from the repo |
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1199 def supportedoutgoingversions(repo): |
30632
7ace5304fec5
changegroup: pass 'repo' to allsupportedversions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
30631
diff
changeset
|
1200 versions = allsupportedversions(repo) |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1201 if 'treemanifest' in repo.requirements: |
27928
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1202 # Versions 01 and 02 support only flat manifests and it's just too |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1203 # expensive to convert between the flat manifest and tree manifest on |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1204 # the fly. Since tree manifests are hashed differently, all of history |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1205 # would have to be converted. Instead, we simply don't even pretend to |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1206 # support versions 01 and 02. |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1207 versions.discard('01') |
c0f11347b107
changegroup: don't support versions 01 and 02 with treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27920
diff
changeset
|
1208 versions.discard('02') |
38874
a232e6744ba3
narrow: move requirement constant from changegroup to repository
Martin von Zweigbergk <martinvonz@google.com>
parents:
38846
diff
changeset
|
1209 if repository.NARROW_REQUIREMENT in repo.requirements: |
36495
94709406f10d
narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36494
diff
changeset
|
1210 # Versions 01 and 02 don't support revlog flags, and we need to |
94709406f10d
narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36494
diff
changeset
|
1211 # support that for stripping and unbundling to work. |
94709406f10d
narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36494
diff
changeset
|
1212 versions.discard('01') |
94709406f10d
narrow: move changegroup.supportedoutgoingversions() override to core
Martin von Zweigbergk <martinvonz@google.com>
parents:
36494
diff
changeset
|
1213 versions.discard('02') |
37135
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1214 if LFS_REQUIREMENT in repo.requirements: |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1215 # Versions 01 and 02 don't support revlog flags, and we need to |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1216 # mark LFS entries with REVIDX_EXTSTORED. |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1217 versions.discard('01') |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1218 versions.discard('02') |
a54113fcc8c9
lfs: move the 'supportedoutgoingversions' handling to changegroup.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
37087
diff
changeset
|
1219 |
27752
29cfc474c5fd
changegroup3: introduce experimental.changegroup3 boolean config
Martin von Zweigbergk <martinvonz@google.com>
parents:
27751
diff
changeset
|
1220 return versions |
27751
a40e2f7fe49d
changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents:
27739
diff
changeset
|
1221 |
33676
91f0677dc920
repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33461
diff
changeset
|
1222 def localversion(repo): |
91f0677dc920
repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33461
diff
changeset
|
1223 # Finds the best version to use for bundles that are meant to be used |
91f0677dc920
repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33461
diff
changeset
|
1224 # locally, such as those from strip and shelve, and temporary bundles. |
91f0677dc920
repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33461
diff
changeset
|
1225 return max(supportedoutgoingversions(repo)) |
91f0677dc920
repair: preserve phase also when not using generaldelta (issue5678)
Martin von Zweigbergk <martinvonz@google.com>
parents:
33461
diff
changeset
|
1226 |
27929
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1227 def safeversion(repo): |
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1228 # Finds the smallest version that it's safe to assume clients of the repo |
27931
1289a122cf3f
shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27929
diff
changeset
|
1229 # will support. For example, all hg versions that support generaldelta also |
1289a122cf3f
shelve: use cg3 for treemanifests
Martin von Zweigbergk <martinvonz@google.com>
parents:
27929
diff
changeset
|
1230 # support changegroup 02. |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1231 versions = supportedoutgoingversions(repo) |
27929
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1232 if 'generaldelta' in repo.requirements: |
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1233 versions.discard('01') |
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1234 assert versions |
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1235 return min(versions) |
3b2ac2115464
changegroup: introduce safeversion()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27928
diff
changeset
|
1236 |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1237 def getbundler(version, repo, bundlecaps=None, filematcher=None): |
27953
88609cfa3745
changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents:
27946
diff
changeset
|
1238 assert version in supportedoutgoingversions(repo) |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1239 |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1240 if filematcher is None: |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1241 filematcher = matchmod.alwaysmatcher(repo.root, '') |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1242 |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1243 if version == '01' and not filematcher.always(): |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1244 raise error.ProgrammingError('version 01 changegroups do not support ' |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1245 'sparse file matchers') |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1246 |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1247 # Requested files could include files not in the local store. So |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1248 # filter those out. |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1249 filematcher = matchmod.intersectmatchers(repo.narrowmatch(), |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1250 filematcher) |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1251 |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1252 return _packermap[version][0](repo, filematcher=filematcher, |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1253 bundlecaps=bundlecaps) |
27751
a40e2f7fe49d
changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents:
27739
diff
changeset
|
1254 |
29593
953839de96ab
bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29371
diff
changeset
|
1255 def getunbundler(version, fh, alg, extras=None): |
953839de96ab
bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29371
diff
changeset
|
1256 return _packermap[version][1](fh, alg, extras=extras) |
27751
a40e2f7fe49d
changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents:
27739
diff
changeset
|
1257 |
20926
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1258 def _changegroupinfo(repo, nodes, source): |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1259 if repo.ui.verbose or source == 'bundle': |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1260 repo.ui.status(_("%d changesets found\n") % len(nodes)) |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1261 if repo.ui.debugflag: |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1262 repo.ui.debug("list of changesets:\n") |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1263 for node in nodes: |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1264 repo.ui.debug("%s\n" % hex(node)) |
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1265 |
34111
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1266 def makechangegroup(repo, outgoing, version, source, fastpath=False, |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1267 bundlecaps=None): |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1268 cgstream = makestream(repo, outgoing, version, source, |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1269 fastpath=fastpath, bundlecaps=bundlecaps) |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1270 return getunbundler(version, util.chunkbuffer(cgstream), None, |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1271 {'clcount': len(outgoing.missing) }) |
f85dfde1731a
changegroup: replace getsubset with makechangegroup
Durham Goode <durham@fb.com>
parents:
34106
diff
changeset
|
1272 |
34118
92f1e2be8ab6
changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents:
34116
diff
changeset
|
1273 def makestream(repo, outgoing, version, source, fastpath=False, |
38834
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1274 bundlecaps=None, filematcher=None): |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1275 bundler = getbundler(version, repo, bundlecaps=bundlecaps, |
1d01cf0416a5
changegroup: move file matcher from narrow extension
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38823
diff
changeset
|
1276 filematcher=filematcher) |
34118
92f1e2be8ab6
changegroup: rename getsubsetraw to makestream
Durham Goode <durham@fb.com>
parents:
34116
diff
changeset
|
1277 |
20925
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1278 repo = repo.unfiltered() |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1279 commonrevs = outgoing.common |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1280 csets = outgoing.missing |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1281 heads = outgoing.missingheads |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1282 # We go through the fast path if we get told to, or if all (unfiltered |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1283 # heads have been requested (since we then know there all linkrevs will |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1284 # be pulled by the client). |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1285 heads.sort() |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1286 fastpathlinkrev = fastpath or ( |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1287 repo.filtername is None and heads == sorted(repo.heads())) |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1288 |
5174c48ed8d8
localrepo: move the _changegroupsubset method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20675
diff
changeset
|
1289 repo.hook('preoutgoing', throw=True, source=source) |
20926
7c1ed40e3325
localrepo: move the changegroupinfo method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20925
diff
changeset
|
1290 _changegroupinfo(repo, csets, source) |
23177
706547a14b8b
changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents:
23168
diff
changeset
|
1291 return bundler.generate(commonrevs, csets, fastpathlinkrev, source) |
706547a14b8b
changegroup: introduce "raw" versions of some commands
Sune Foldager <cryo@cyanite.org>
parents:
23168
diff
changeset
|
1292 |
28361
277a22cd8741
changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents:
28360
diff
changeset
|
1293 def _addchangegroupfiles(repo, source, revmap, trp, expectedfiles, needfiles): |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1294 revisions = 0 |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1295 files = 0 |
38388
daa08d45740f
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38379
diff
changeset
|
1296 progress = repo.ui.makeprogress(_('files'), unit=_('files'), |
daa08d45740f
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38379
diff
changeset
|
1297 total=expectedfiles) |
29736
4e7be6e33269
changegroup: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents:
29704
diff
changeset
|
1298 for chunkdata in iter(source.filelogheader, {}): |
28361
277a22cd8741
changegroup: progress for added files is not measured in "chunks"
Martin von Zweigbergk <martinvonz@google.com>
parents:
28360
diff
changeset
|
1299 files += 1 |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1300 f = chunkdata["filename"] |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1301 repo.ui.debug("adding %s revisions\n" % f) |
38388
daa08d45740f
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38379
diff
changeset
|
1302 progress.increment() |
27754
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
1303 fl = repo.file(f) |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1304 o = len(fl) |
24120
a450e0a2ba0a
revlog: in addgroup, reject ill-formed deltas based on censored nodes
Mike Edgar <adgar@google.com>
parents:
23897
diff
changeset
|
1305 try: |
34298
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
1306 deltas = source.deltaiter() |
1db9abf407c5
revlog: add revmap back to revlog.addgroup
Durham Goode <durham@fb.com>
parents:
34264
diff
changeset
|
1307 if not fl.addgroup(deltas, revmap, trp): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
1308 raise error.Abort(_("received file revlog group is empty")) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25624
diff
changeset
|
1309 except error.CensoredBaseError as e: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
1310 raise error.Abort(_("received delta base is censored: %s") % e) |
27754
a09f143daaf4
changegroup3: move treemanifest support into _unpackmanifests()
Martin von Zweigbergk <martinvonz@google.com>
parents:
27753
diff
changeset
|
1311 revisions += len(fl) - o |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1312 if f in needfiles: |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1313 needs = needfiles[f] |
38823
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38557
diff
changeset
|
1314 for new in pycompat.xrange(o, len(fl)): |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1315 n = fl.node(new) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1316 if n in needs: |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1317 needs.remove(n) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1318 else: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
1319 raise error.Abort( |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1320 _("received spurious file revlog entry")) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1321 if not needs: |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1322 del needfiles[f] |
38388
daa08d45740f
changegroup: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
38379
diff
changeset
|
1323 progress.complete() |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1324 |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1325 for f, needs in needfiles.iteritems(): |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1326 fl = repo.file(f) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1327 for n in needs: |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1328 try: |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1329 fl.rev(n) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1330 except error.LookupError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26540
diff
changeset
|
1331 raise error.Abort( |
20932
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1332 _('missing file data for %s:%s - run hg verify') % |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1333 (f, hex(n))) |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1334 |
0ac83e4e4f7c
localrepo: move the addchangegroupfiles method in changegroup module
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20931
diff
changeset
|
1335 return revisions, files |
38923
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1336 |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1337 def _packellipsischangegroup(repo, common, match, relevant_nodes, |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1338 ellipsisroots, visitnodes, depth, source, version): |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1339 if version in ('01', '02'): |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1340 raise error.Abort( |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1341 'ellipsis nodes require at least cg3 on client and server, ' |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1342 'but negotiated version %s' % version) |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1343 # We wrap cg1packer.revchunk, using a side channel to pass |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1344 # relevant_nodes into that area. Then if linknode isn't in the |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1345 # set, we know we have an ellipsis node and we should defer |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1346 # sending that node's data. We override close() to detect |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1347 # pending ellipsis nodes and flush them. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1348 packer = getbundler(version, repo, filematcher=match) |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1349 # Give the packer the list of nodes which should not be |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1350 # ellipsis nodes. We store this rather than the set of nodes |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1351 # that should be an ellipsis because for very large histories |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1352 # we expect this to be significantly smaller. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1353 packer.full_nodes = relevant_nodes |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1354 # Maps ellipsis revs to their roots at the changelog level. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1355 packer.precomputed_ellipsis = ellipsisroots |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1356 # Maps CL revs to per-revlog revisions. Cleared in close() at |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1357 # the end of each group. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1358 packer.clrev_to_localrev = {} |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1359 packer.next_clrev_to_localrev = {} |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1360 # Maps changelog nodes to changelog revs. Filled in once |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1361 # during changelog stage and then left unmodified. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1362 packer.clnode_to_rev = {} |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1363 packer.changelog_done = False |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1364 # If true, informs the packer that it is serving shallow content and might |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1365 # need to pack file contents not introduced by the changes being packed. |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1366 packer.is_shallow = depth is not None |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1367 |
a2f521773761
changegroup: move _packellipsischangegroup() from narrow
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38922
diff
changeset
|
1368 return packer.generate(common, visitnodes, False, source) |