Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/streamclone.py @ 26467:ff2c89239d49

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
streamclone: teach canperformstreamclone to be bundle2 aware We add an argument to canperformstreamclone() to return False if a bundle2 stream clone is available. This will enable the legacy stream clone step to no-op when a bundle2 stream clone is supported. The commented code will be made active when bundle2 supports streaming clone. This patch does foreshadow the introduction of the "stream" bundle2 capability and its "v1" sub-capability. The bundle2 capability mirrors the existing "stream" capability and is needed so clients know whether a server explicitly supports streaming clones over bundle2 (servers up to this point support bundle2 without streaming clone support). The sub-capability will denote which data formats and variations are supported. Currently, the value "v1" denotes the existing streaming clone data format, which I intend to reuse inside a bundle2 part. My intent is to eventually introduce alternate data formats that can be produced and consumed more efficiently. Having a sub-capability means we don't need to introduce a new top-level bundle2 capability when new formats are introduced. This doesn't really have any implications beyond making the capabilities namespace more organized.
author Gregory Szorc <gregory.szorc@gmail.com>
date Sun, 04 Oct 2015 18:35:19 -0700
parents 3515db5aae05
children 19bbd53af46d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # streamclone.py - producing and consuming streaming repository data
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2015 Gregory Szorc <gregory.szorc@gmail.com>
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 from __future__ import absolute_import
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
10 import time
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
11
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
12 from .i18n import _
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 from . import (
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 branchmap,
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
15 error,
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
16 store,
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
17 util,
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 )
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
20 def canperformstreamclone(pullop, bailifbundle2supported=False):
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
21 """Whether it is possible to perform a streaming clone as part of pull.
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
22
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
23 ``bailifbundle2supported`` will cause the function to return False if
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
24 bundle2 stream clones are supported. It should only be called by the
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
25 legacy stream clone code path.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
26
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
27 Returns a tuple of (supported, requirements). ``supported`` is True if
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
28 streaming clone is supported and False otherwise. ``requirements`` is
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
29 a set of repo requirements from the remote, or ``None`` if stream clone
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
30 isn't supported.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
31 """
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
32 repo = pullop.repo
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
33 remote = pullop.remote
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
34
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
35 bundle2supported = False
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
36 if pullop.canusebundle2:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
37 if 'v1' in pullop.remotebundle2caps.get('stream', []):
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
38 bundle2supported = True
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
39 # else
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
40 # Server doesn't support bundle2 stream clone or doesn't support
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
41 # the versions we support. Fall back and possibly allow legacy.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
42
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
43 # Ensures legacy code path uses available bundle2.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
44 if bailifbundle2supported and bundle2supported:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
45 return False, None
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
46 # Ensures bundle2 doesn't try to do a stream clone if it isn't supported.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
47 #elif not bailifbundle2supported and not bundle2supported:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
48 # return False, None
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
49
26447
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
50 # Streaming clone only works on empty repositories.
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
51 if len(repo):
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
52 return False, None
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
53
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
54 # Streaming clone only works if all data is being requested.
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
55 if pullop.heads:
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
56 return False, None
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
57
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
58 streamrequested = pullop.streamclonerequested
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
59
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
60 # If we don't have a preference, let the server decide for us. This
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
61 # likely only comes into play in LANs.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
62 if streamrequested is None:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
63 # The server can advertise whether to prefer streaming clone.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
64 streamrequested = remote.capable('stream-preferred')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
65
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
66 if not streamrequested:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
67 return False, None
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
68
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
69 # In order for stream clone to work, the client has to support all the
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
70 # requirements advertised by the server.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
71 #
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
72 # The server advertises its requirements via the "stream" and "streamreqs"
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
73 # capability. "stream" (a value-less capability) is advertised if and only
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
74 # if the only requirement is "revlogv1." Else, the "streamreqs" capability
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
75 # is advertised and contains a comma-delimited list of requirements.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
76 requirements = set()
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
77 if remote.capable('stream'):
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
78 requirements.add('revlogv1')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
79 else:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
80 streamreqs = remote.capable('streamreqs')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
81 # This is weird and shouldn't happen with modern servers.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
82 if not streamreqs:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
83 return False, None
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
84
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
85 streamreqs = set(streamreqs.split(','))
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
86 # Server requires something we don't support. Bail.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
87 if streamreqs - repo.supportedformats:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
88 return False, None
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
89 requirements = streamreqs
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
90
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
91 return True, requirements
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
92
26462
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
93 def maybeperformlegacystreamclone(pullop):
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
94 """Possibly perform a legacy stream clone operation.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
95
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
96 Legacy stream clones are performed as part of pull but before all other
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
97 operations.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
98
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
99 A legacy stream clone will not be performed if a bundle2 stream clone is
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
100 supported.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
101 """
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
102 supported, requirements = canperformstreamclone(pullop)
26458
362793295640 streamclone: refactor maybeperformstreamclone to take a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26447
diff changeset
103
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
104 if not supported:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
105 return
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
106
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
107 repo = pullop.repo
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
108 remote = pullop.remote
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
109
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
110 # Save remote branchmap. We will use it later to speed up branchcache
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
111 # creation.
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
112 rbranchmap = None
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
113 if remote.capable('branchmap'):
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
114 rbranchmap = remote.branchmap()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
115
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
116 fp = remote.stream_out()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
117 l = fp.readline()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
118 try:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
119 resp = int(l)
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
120 except ValueError:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
121 raise error.ResponseError(
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
122 _('unexpected response from remote server:'), l)
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
123 if resp == 1:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
124 raise util.Abort(_('operation forbidden by server'))
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
125 elif resp == 2:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
126 raise util.Abort(_('locking the remote repository failed'))
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
127 elif resp != 0:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
128 raise util.Abort(_('the server sent an unknown error code'))
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
129
26461
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
130 lock = repo.lock()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
131 try:
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
132 consumev1(repo, fp)
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
133
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
134 # new requirements = old non-format requirements +
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
135 # new format-related remote requirements
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
136 # requirements from the streamed-in repository
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
137 repo.requirements = requirements | (
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
138 repo.requirements - repo.supportedformats)
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
139 repo._applyopenerreqs()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
140 repo._writerequirements()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
141
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
142 if rbranchmap:
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
143 branchmap.replacecache(repo, rbranchmap)
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
144
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
145 repo.invalidate()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
146 finally:
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
147 lock.release()
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
148
26444
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
149 def allowservergeneration(ui):
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
150 """Whether streaming clones are allowed from the server."""
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
151 return ui.configbool('server', 'uncompressed', True, untrusted=True)
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
152
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
153 # This is it's own function so extensions can override it.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
154 def _walkstreamfiles(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
155 return repo.store.walk()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
156
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
157 def generatev1(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
158 """Emit content for version 1 of a streaming clone.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
159
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
160 This is a generator of raw chunks that constitute a streaming clone.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
161
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
162 The stream begins with a line of 2 space-delimited integers containing the
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
163 number of entries and total bytes size.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
164
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
165 Next, are N entries for each file being transferred. Each file entry starts
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
166 as a line with the file name and integer size delimited by a null byte.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
167 The raw file data follows. Following the raw file data is the next file
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
168 entry, or EOF.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
169
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
170 When used on the wire protocol, an additional line indicating protocol
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
171 success will be prepended to the stream. This function is not responsible
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
172 for adding it.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
173
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
174 This function will obtain a repository lock to ensure a consistent view of
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
175 the store is captured. It therefore may raise LockError.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
176 """
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
177 entries = []
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
178 total_bytes = 0
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
179 # Get consistent snapshot of repo, lock during scan.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
180 lock = repo.lock()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
181 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
182 repo.ui.debug('scanning\n')
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
183 for name, ename, size in _walkstreamfiles(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
184 if size:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
185 entries.append((name, size))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
186 total_bytes += size
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
187 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
188 lock.release()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
189
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
190 repo.ui.debug('%d files, %d bytes to transfer\n' %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
191 (len(entries), total_bytes))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
192 yield '%d %d\n' % (len(entries), total_bytes)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
193
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
194 svfs = repo.svfs
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
195 oldaudit = svfs.mustaudit
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
196 debugflag = repo.ui.debugflag
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
197 svfs.mustaudit = False
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
198
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
199 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
200 for name, size in entries:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
201 if debugflag:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
202 repo.ui.debug('sending %s (%d bytes)\n' % (name, size))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
203 # partially encode name over the wire for backwards compat
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
204 yield '%s\0%d\n' % (store.encodedir(name), size)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
205 if size <= 65536:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
206 fp = svfs(name)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
207 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
208 data = fp.read(size)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
209 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
210 fp.close()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
211 yield data
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
212 else:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
213 for chunk in util.filechunkiter(svfs(name), limit=size):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
214 yield chunk
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
215 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
216 svfs.mustaudit = oldaudit
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
217
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
218 def consumev1(repo, fp):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
219 """Apply the contents from version 1 of a streaming clone file handle.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
220
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
221 This takes the output from "streamout" and applies it to the specified
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
222 repository.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
223
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
224 Like "streamout," the status line added by the wire protocol is not handled
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
225 by this function.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
226 """
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
227 lock = repo.lock()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
228 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
229 repo.ui.status(_('streaming all changes\n'))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
230 l = fp.readline()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
231 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
232 total_files, total_bytes = map(int, l.split(' ', 1))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
233 except (ValueError, TypeError):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
234 raise error.ResponseError(
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
235 _('unexpected response from remote server:'), l)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
236 repo.ui.status(_('%d files to transfer, %s of data\n') %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
237 (total_files, util.bytecount(total_bytes)))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
238 handled_bytes = 0
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
239 repo.ui.progress(_('clone'), 0, total=total_bytes)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
240 start = time.time()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
241
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
242 tr = repo.transaction(_('clone'))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
243 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
244 for i in xrange(total_files):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
245 # XXX doesn't support '\n' or '\r' in filenames
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
246 l = fp.readline()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
247 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
248 name, size = l.split('\0', 1)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
249 size = int(size)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
250 except (ValueError, TypeError):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
251 raise error.ResponseError(
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
252 _('unexpected response from remote server:'), l)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
253 if repo.ui.debugflag:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
254 repo.ui.debug('adding %s (%s)\n' %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
255 (name, util.bytecount(size)))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
256 # for backwards compat, name was partially encoded
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
257 ofp = repo.svfs(store.decodedir(name), 'w')
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
258 for chunk in util.filechunkiter(fp, limit=size):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
259 handled_bytes += len(chunk)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
260 repo.ui.progress(_('clone'), handled_bytes,
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
261 total=total_bytes)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
262 ofp.write(chunk)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
263 ofp.close()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
264 tr.close()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
265 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
266 tr.release()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
267
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
268 # Writing straight to files circumvented the inmemory caches
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
269 repo.invalidate()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
270
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
271 elapsed = time.time() - start
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
272 if elapsed <= 0:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
273 elapsed = 0.001
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
274 repo.ui.progress(_('clone'), None)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
275 repo.ui.status(_('transferred %s in %.1f seconds (%s/sec)\n') %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
276 (util.bytecount(total_bytes), elapsed,
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
277 util.bytecount(total_bytes / elapsed)))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
278 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
279 lock.release()