annotate hgext/largefiles/lfcommands.py @ 15793:3ef07ecdb0d5

largefiles: correctly handle dirstate status when rebasing When rebasing, we need to trust that the standins are always correct. The rebase operation updates the standins according to the changeset it is rebasing. We need to make the largefiles in the working copy match. If we don't make them match, then they get accidentally reverted, either during the rebase or during the next commit after the rebase. This worked previously only becuase we were relying on the behavior that largefiles with a changed standin, but unchanged contents, never showed up in the list of modified largefiles. Unfortunately, pre-commit hooks can get an incorrect status this way, and it also results in extra execution of code. The solution is to simply trust the standins when we are about to commit a rebased changeset, and politely ask updatelfiles() to pull the new contents down. In this case, updatelfiles() will also mark any files it has pulled down as dirty in the lfdirstate so that pre-commit hooks will get correct status output.
author Na'Tosha Bard <natosha@unity3d.com>
date Sat, 07 Jan 2012 18:43:34 +0100
parents 9036c7d106bf
children 62098aeb1e15
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
1 # Copyright 2009-2010 Gregory P. Ward
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
3 # Copyright 2010-2011 Fog Creek Software
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
4 # Copyright 2010-2011 Unity Technologies
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
5 #
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
8
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
9 '''High-level command function for lfconvert, plus the cmdtable.'''
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
10
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
11 import os
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
12 import shutil
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
13
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
14 from mercurial import util, match as match_, hg, node, context, error
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
15 from mercurial.i18n import _
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
16
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
17 import lfutil
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
18 import basestore
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
19
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
20 # -- Commands ----------------------------------------------------------
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
21
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
22 def lfconvert(ui, src, dest, *pats, **opts):
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
23 '''convert a normal repository to a largefiles repository
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
24
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
25 Convert repository SOURCE to a new repository DEST, identical to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
26 SOURCE except that certain files will be converted as largefiles:
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
27 specifically, any file that matches any PATTERN *or* whose size is
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
28 above the minimum size threshold is converted as a largefile. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
29 size used to determine whether or not to track a file as a
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
30 largefile is the size of the first version of the file. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
31 minimum size can be specified either with --size or in
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
32 configuration as ``largefiles.size``.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
33
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
34 After running this command you will need to make sure that
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
35 largefiles is enabled anywhere you intend to push the new
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
36 repository.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
37
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
38 Use --to-normal to convert largefiles back to normal files; after
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
39 this, the DEST repository can be used without largefiles at all.'''
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
40
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
41 if opts['to_normal']:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
42 tolfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
43 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
44 tolfile = True
15227
a7686abf73a6 largefiles: factor out lfutil.getminsize()
Greg Ward <greg@gerg.ca>
parents: 15224
diff changeset
45 size = lfutil.getminsize(ui, True, opts.get('size'), default=None)
15340
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
46
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
47 if not hg.islocal(src):
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
48 raise util.Abort(_('%s is not a local Mercurial repo') % src)
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
49 if not hg.islocal(dest):
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
50 raise util.Abort(_('%s is not a local Mercurial repo') % dest)
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
51
15339
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
52 rsrc = hg.repository(ui, src)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
53 ui.status(_('initializing destination %s\n') % dest)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
54 rdst = hg.repository(ui, dest, create=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
55
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
56 success = False
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
57 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
58 # Lock destination to prevent modification while it is converted to.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
59 # Don't need to lock src because we are just reading from its history
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
60 # which can't change.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
61 dst_lock = rdst.lock()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
62
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
63 # Get a list of all changesets in the source. The easy way to do this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
64 # is to simply walk the changelog, using changelog.nodesbewteen().
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
65 # Take a look at mercurial/revlog.py:639 for more details.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
66 # Use a generator instead of a list to decrease memory usage
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
67 ctxs = (rsrc[ctx] for ctx in rsrc.changelog.nodesbetween(None,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
68 rsrc.heads())[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
69 revmap = {node.nullid: node.nullid}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
70 if tolfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
71 lfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
72 normalfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
73 if not pats:
15579
6c5e6ebe0812 largefiles: use "ui.configlist()" to get largefiles.patterns configuration
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15472
diff changeset
74 pats = ui.configlist(lfutil.longname, 'patterns', default=[])
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
75 if pats:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
76 matcher = match_.match(rsrc.root, '', list(pats))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
77 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
78 matcher = None
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
79
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
80 lfiletohash = {}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
81 for ctx in ctxs:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
82 ui.progress(_('converting revisions'), ctx.rev(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
83 unit=_('revision'), total=rsrc['tip'].rev())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
84 _lfconvert_addchangeset(rsrc, rdst, ctx, revmap,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
85 lfiles, normalfiles, matcher, size, lfiletohash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
86 ui.progress(_('converting revisions'), None)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
87
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
88 if os.path.exists(rdst.wjoin(lfutil.shortname)):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
89 shutil.rmtree(rdst.wjoin(lfutil.shortname))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
90
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
91 for f in lfiletohash.keys():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
92 if os.path.isfile(rdst.wjoin(f)):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
93 os.unlink(rdst.wjoin(f))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
94 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
95 os.removedirs(os.path.dirname(rdst.wjoin(f)))
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
96 except OSError:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
97 pass
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
98
15303
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
99 # If there were any files converted to largefiles, add largefiles
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
100 # to the destination repository's requirements.
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
101 if lfiles:
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
102 rdst.requirements.add('largefiles')
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
103 rdst._writerequirements()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
104 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
105 for ctx in ctxs:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
106 ui.progress(_('converting revisions'), ctx.rev(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
107 unit=_('revision'), total=rsrc['tip'].rev())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
108 _addchangeset(ui, rsrc, rdst, ctx, revmap)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
109
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
110 ui.progress(_('converting revisions'), None)
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
111 success = True
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
112 finally:
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
113 if not success:
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
114 # we failed, remove the new directory
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
115 shutil.rmtree(rdst.root)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
116 dst_lock.release()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
117
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
118 def _addchangeset(ui, rsrc, rdst, ctx, revmap):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
119 # Convert src parents to dst parents
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
120 parents = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
121 for p in ctx.parents():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
122 parents.append(revmap[p.node()])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
123 while len(parents) < 2:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
124 parents.append(node.nullid)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
125
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
126 # Generate list of changed files
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
127 files = set(ctx.files())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
128 if node.nullid not in parents:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
129 mc = ctx.manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
130 mp1 = ctx.parents()[0].manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
131 mp2 = ctx.parents()[1].manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
132 files |= (set(mp1) | set(mp2)) - set(mc)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
133 for f in mc:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
134 if mc[f] != mp1.get(f, None) or mc[f] != mp2.get(f, None):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
135 files.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
136
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
137 def getfilectx(repo, memctx, f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
138 if lfutil.standin(f) in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
139 # if the file isn't in the manifest then it was removed
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
140 # or renamed, raise IOError to indicate this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
141 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
142 fctx = ctx.filectx(lfutil.standin(f))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
143 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
144 raise IOError()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
145 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
146 if renamed:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
147 renamed = lfutil.splitstandin(renamed[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
148
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
149 hash = fctx.data().strip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
150 path = lfutil.findfile(rsrc, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
151 ### TODO: What if the file is not cached?
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
152 data = ''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
153 fd = None
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
154 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
155 fd = open(path, 'rb')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
156 data = fd.read()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
157 finally:
15172
fb1dcd2aae2a largefiles: fix multistatement line
Matt Mackall <mpm@selenic.com>
parents: 15171
diff changeset
158 if fd:
fb1dcd2aae2a largefiles: fix multistatement line
Matt Mackall <mpm@selenic.com>
parents: 15171
diff changeset
159 fd.close()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
160 return context.memfilectx(f, data, 'l' in fctx.flags(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
161 'x' in fctx.flags(), renamed)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
162 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
163 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
164 fctx = ctx.filectx(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
165 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
166 raise IOError()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
167 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
168 if renamed:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
169 renamed = renamed[0]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
170 data = fctx.data()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
171 if f == '.hgtags':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
172 newdata = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
173 for line in data.splitlines():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
174 id, name = line.split(' ', 1)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
175 newdata.append('%s %s\n' % (node.hex(revmap[node.bin(id)]),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
176 name))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
177 data = ''.join(newdata)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
178 return context.memfilectx(f, data, 'l' in fctx.flags(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
179 'x' in fctx.flags(), renamed)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
180
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
181 dstfiles = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
182 for file in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
183 if lfutil.isstandin(file):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
184 dstfiles.append(lfutil.splitstandin(file))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
185 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
186 dstfiles.append(file)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
187 # Commit
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
188 mctx = context.memctx(rdst, parents, ctx.description(), dstfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
189 getfilectx, ctx.user(), ctx.date(), ctx.extra())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
190 ret = rdst.commitctx(mctx)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
191 rdst.dirstate.setparents(ret)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
192 revmap[ctx.node()] = rdst.changelog.tip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
193
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
194 def _lfconvert_addchangeset(rsrc, rdst, ctx, revmap, lfiles, normalfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
195 matcher, size, lfiletohash):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
196 # Convert src parents to dst parents
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
197 parents = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
198 for p in ctx.parents():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
199 parents.append(revmap[p.node()])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
200 while len(parents) < 2:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
201 parents.append(node.nullid)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
202
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
203 # Generate list of changed files
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
204 files = set(ctx.files())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
205 if node.nullid not in parents:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
206 mc = ctx.manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
207 mp1 = ctx.parents()[0].manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
208 mp2 = ctx.parents()[1].manifest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
209 files |= (set(mp1) | set(mp2)) - set(mc)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
210 for f in mc:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
211 if mc[f] != mp1.get(f, None) or mc[f] != mp2.get(f, None):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
212 files.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
213
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
214 dstfiles = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
215 for f in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
216 if f not in lfiles and f not in normalfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
217 islfile = _islfile(f, ctx, matcher, size)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
218 # If this file was renamed or copied then copy
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
219 # the lfileness of its predecessor
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
220 if f in ctx.manifest():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
221 fctx = ctx.filectx(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
222 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
223 renamedlfile = renamed and renamed[0] in lfiles
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
224 islfile |= renamedlfile
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
225 if 'l' in fctx.flags():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
226 if renamedlfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
227 raise util.Abort(
15380
a53888685a6c largefiles: fix uppercase in abort message
Martin Geisler <mg@aragost.com>
parents: 15371
diff changeset
228 _('renamed/copied largefile %s becomes symlink')
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
229 % f)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
230 islfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
231 if islfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
232 lfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
233 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
234 normalfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
235
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
236 if f in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
237 dstfiles.append(lfutil.standin(f))
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
238 # largefile in manifest if it has not been removed/renamed
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
239 if f in ctx.manifest():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
240 if 'l' in ctx.filectx(f).flags():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
241 if renamed and renamed[0] in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
242 raise util.Abort(_('largefile %s becomes symlink') % f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
243
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
244 # largefile was modified, update standins
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
245 fullpath = rdst.wjoin(f)
15371
f26ed4ea46d8 largefiles: remove lfutil.createdir, replace calls with util.makedirs
Hao Lian <hao@fogcreek.com>
parents: 15340
diff changeset
246 util.makedirs(os.path.dirname(fullpath))
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
247 m = util.sha1('')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
248 m.update(ctx[f].data())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
249 hash = m.hexdigest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
250 if f not in lfiletohash or lfiletohash[f] != hash:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
251 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
252 fd = open(fullpath, 'wb')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
253 fd.write(ctx[f].data())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
254 finally:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
255 if fd:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
256 fd.close()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
257 executable = 'x' in ctx[f].flags()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
258 os.chmod(fullpath, lfutil.getmode(executable))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
259 lfutil.writestandin(rdst, lfutil.standin(f), hash,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
260 executable)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
261 lfiletohash[f] = hash
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
262 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
263 # normal file
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
264 dstfiles.append(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
265
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
266 def getfilectx(repo, memctx, f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
267 if lfutil.isstandin(f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
268 # if the file isn't in the manifest then it was removed
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
269 # or renamed, raise IOError to indicate this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
270 srcfname = lfutil.splitstandin(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
271 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
272 fctx = ctx.filectx(srcfname)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
273 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
274 raise IOError()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
275 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
276 if renamed:
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
277 # standin is always a largefile because largefile-ness
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
278 # doesn't change after rename or copy
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
279 renamed = lfutil.standin(renamed[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
280
15313
3eb1a90ea409 largefiles: fix newline for lfconverted repos
Eli Carter <eli.carter@tektronix.com>
parents: 15303
diff changeset
281 return context.memfilectx(f, lfiletohash[srcfname] + '\n', 'l' in
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
282 fctx.flags(), 'x' in fctx.flags(), renamed)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
283 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
284 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
285 fctx = ctx.filectx(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
286 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
287 raise IOError()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
288 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
289 if renamed:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
290 renamed = renamed[0]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
291
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
292 data = fctx.data()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
293 if f == '.hgtags':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
294 newdata = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
295 for line in data.splitlines():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
296 id, name = line.split(' ', 1)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
297 newdata.append('%s %s\n' % (node.hex(revmap[node.bin(id)]),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
298 name))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
299 data = ''.join(newdata)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
300 return context.memfilectx(f, data, 'l' in fctx.flags(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
301 'x' in fctx.flags(), renamed)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
302
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
303 # Commit
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
304 mctx = context.memctx(rdst, parents, ctx.description(), dstfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
305 getfilectx, ctx.user(), ctx.date(), ctx.extra())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
306 ret = rdst.commitctx(mctx)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
307 rdst.dirstate.setparents(ret)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
308 revmap[ctx.node()] = rdst.changelog.tip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
309
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
310 def _islfile(file, ctx, matcher, size):
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
311 '''Return true if file should be considered a largefile, i.e.
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
312 matcher matches it or it is larger than size.'''
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
313 # never store special .hg* files as largefiles
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
314 if file == '.hgtags' or file == '.hgignore' or file == '.hgsigs':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
315 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
316 if matcher and matcher(file):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
317 return True
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
318 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
319 return ctx.filectx(file).size() >= size * 1024 * 1024
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
320 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
321 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
322
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
323 def uploadlfiles(ui, rsrc, rdst, files):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
324 '''upload largefiles to the central store'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
325
15317
41f371150ccb largefiles: make the store primary, and the user cache secondary
Benjamin Pollack <benjamin@bitquabit.com>
parents: 15313
diff changeset
326 if not files:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
327 return
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
328
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
329 store = basestore._openstore(rsrc, rdst, put=True)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
330
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
331 at = 0
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
332 files = filter(lambda h: not store.exists(h), files)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
333 for hash in files:
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
334 ui.progress(_('uploading largefiles'), at, unit='largefile',
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
335 total=len(files))
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
336 source = lfutil.findfile(rsrc, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
337 if not source:
15253
67d010779907 largefiles: improve error reporting
Greg Ward <greg@gerg.ca>
parents: 15252
diff changeset
338 raise util.Abort(_('largefile %s missing from store'
67d010779907 largefiles: improve error reporting
Greg Ward <greg@gerg.ca>
parents: 15252
diff changeset
339 ' (needs to be uploaded)') % hash)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
340 # XXX check for errors here
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
341 store.put(source, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
342 at += 1
15173
3d27a8ff895f largefiles: mark a string for translation
Matt Mackall <mpm@selenic.com>
parents: 15172
diff changeset
343 ui.progress(_('uploading largefiles'), None)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
344
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
345 def verifylfiles(ui, repo, all=False, contents=False):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
346 '''Verify that every big file revision in the current changeset
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
347 exists in the central store. With --contents, also verify that
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
348 the contents of each big file revision are correct (SHA-1 hash
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
349 matches the revision ID). With --all, check every changeset in
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
350 this repository.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
351 if all:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
352 # Pass a list to the function rather than an iterator because we know a
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
353 # list will work.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
354 revs = range(len(repo))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
355 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
356 revs = ['.']
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
357
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
358 store = basestore._openstore(repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
359 return store.verify(revs, contents=contents)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
360
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
361 def cachelfiles(ui, repo, node):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
362 '''cachelfiles ensures that all largefiles needed by the specified revision
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
363 are present in the repository's largefile cache.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
364
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
365 returns a tuple (cached, missing). cached is the list of files downloaded
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
366 by this operation; missing is the list of files that were needed but could
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
367 not be found.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
368 lfiles = lfutil.listlfiles(repo, node)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
369 toget = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
370
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
371 for lfile in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
372 expectedhash = repo[node][lfutil.standin(lfile)].data().strip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
373 # if it exists and its hash matches, it might have been locally
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
374 # modified before updating and the user chose 'local'. in this case,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
375 # it will not be in any store, so don't look for it.
15255
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
376 if ((not os.path.exists(repo.wjoin(lfile)) or
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
377 expectedhash != lfutil.hashfile(repo.wjoin(lfile))) and
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
378 not lfutil.findfile(repo, expectedhash)):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
379 toget.append((lfile, expectedhash))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
380
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
381 if toget:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
382 store = basestore._openstore(repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
383 ret = store.get(toget)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
384 return ret
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
385
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
386 return ([], [])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
387
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
388 def updatelfiles(ui, repo, filelist=None, printmessage=True):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
389 wlock = repo.wlock()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
390 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
391 lfdirstate = lfutil.openlfdirstate(ui, repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
392 lfiles = set(lfutil.listlfiles(repo)) | set(lfdirstate)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
393
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
394 if filelist is not None:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
395 lfiles = [f for f in lfiles if f in filelist]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
396
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
397 printed = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
398 if printmessage and lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
399 ui.status(_('getting changed largefiles\n'))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
400 printed = True
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
401 cachelfiles(ui, repo, '.')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
402
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
403 updated, removed = 0, 0
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
404 for i in map(lambda f: _updatelfile(repo, lfdirstate, f), lfiles):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
405 # increment the appropriate counter according to _updatelfile's
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
406 # return value
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
407 updated += i > 0 and i or 0
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
408 removed -= i < 0 and i or 0
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
409 if printmessage and (removed or updated) and not printed:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
410 ui.status(_('getting changed largefiles\n'))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
411 printed = True
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
412
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
413 lfdirstate.write()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
414 if printed and printmessage:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
415 ui.status(_('%d largefiles updated, %d removed\n') % (updated,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
416 removed))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
417 finally:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
418 wlock.release()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
419
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
420 def _updatelfile(repo, lfdirstate, lfile):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
421 '''updates a single largefile and copies the state of its standin from
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
422 the repository's dirstate to its state in the lfdirstate.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
423
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
424 returns 1 if the file was modified, -1 if the file was removed, 0 if the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
425 file was unchanged, and None if the needed largefile was missing from the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
426 cache.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
427 ret = 0
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
428 abslfile = repo.wjoin(lfile)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
429 absstandin = repo.wjoin(lfutil.standin(lfile))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
430 if os.path.exists(absstandin):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
431 if os.path.exists(absstandin+'.orig'):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
432 shutil.copyfile(abslfile, abslfile+'.orig')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
433 expecthash = lfutil.readstandin(repo, lfile)
15255
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
434 if (expecthash != '' and
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
435 (not os.path.exists(abslfile) or
7ab05d752405 largefiles: cosmetics, whitespace, code style
Greg Ward <greg@gerg.ca>
parents: 15254
diff changeset
436 expecthash != lfutil.hashfile(abslfile))):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
437 if not lfutil.copyfromcache(repo, expecthash, lfile):
15472
6a7e874390b0 largefiles: treat status of cache missed largefiles as "missing" correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15380
diff changeset
438 # use normallookup() to allocate entry in largefiles dirstate,
6a7e874390b0 largefiles: treat status of cache missed largefiles as "missing" correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15380
diff changeset
439 # because lack of it misleads lfiles_repo.status() into
6a7e874390b0 largefiles: treat status of cache missed largefiles as "missing" correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15380
diff changeset
440 # recognition that such cache missing files are REMOVED.
6a7e874390b0 largefiles: treat status of cache missed largefiles as "missing" correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15380
diff changeset
441 lfdirstate.normallookup(lfile)
6a7e874390b0 largefiles: treat status of cache missed largefiles as "missing" correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15380
diff changeset
442 return None # don't try to set the mode
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
443 ret = 1
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
444 mode = os.stat(absstandin).st_mode
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
445 if mode != os.stat(abslfile).st_mode:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
446 os.chmod(abslfile, mode)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
447 ret = 1
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
448 else:
15663
9036c7d106bf largefiles: handle merges between normal files and largefiles (issue3084)
Martin Geisler <mg@aragost.com>
parents: 15579
diff changeset
449 # Remove lfiles for which the standin is deleted, unless the
9036c7d106bf largefiles: handle merges between normal files and largefiles (issue3084)
Martin Geisler <mg@aragost.com>
parents: 15579
diff changeset
450 # lfile is added to the repository again. This happens when a
9036c7d106bf largefiles: handle merges between normal files and largefiles (issue3084)
Martin Geisler <mg@aragost.com>
parents: 15579
diff changeset
451 # largefile is converted back to a normal file: the standin
9036c7d106bf largefiles: handle merges between normal files and largefiles (issue3084)
Martin Geisler <mg@aragost.com>
parents: 15579
diff changeset
452 # disappears, but a new (normal) file appears as the lfile.
9036c7d106bf largefiles: handle merges between normal files and largefiles (issue3084)
Martin Geisler <mg@aragost.com>
parents: 15579
diff changeset
453 if os.path.exists(abslfile) and lfile not in repo[None]:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
454 os.unlink(abslfile)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
455 ret = -1
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
456 state = repo.dirstate[lfutil.standin(lfile)]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
457 if state == 'n':
15793
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
458 # When rebasing, we need to synchronize the standin and the largefile,
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
459 # because otherwise the largefile will get reverted. But for commit's
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
460 # sake, we have to mark the file as unclean.
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
461 if getattr(repo, "_isrebasing", False):
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
462 lfdirstate.normallookup(lfile)
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
463 else:
3ef07ecdb0d5 largefiles: correctly handle dirstate status when rebasing
Na'Tosha Bard <natosha@unity3d.com>
parents: 15663
diff changeset
464 lfdirstate.normal(lfile)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
465 elif state == 'r':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
466 lfdirstate.remove(lfile)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
467 elif state == 'a':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
468 lfdirstate.add(lfile)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
469 elif state == '?':
15224
7c604d8c7e83 largefiles: remove pre-1.9 code from extension first bundled with 1.9
Na'Tosha Bard <natosha@unity3d.com>
parents: 15173
diff changeset
470 lfdirstate.drop(lfile)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
471 return ret
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
472
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
473 # -- hg commands declarations ------------------------------------------------
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
474
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
475 cmdtable = {
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
476 'lfconvert': (lfconvert,
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
477 [('s', 'size', '',
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
478 _('minimum size (MB) for files to be converted '
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
479 'as largefiles'),
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
480 'SIZE'),
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
481 ('', 'to-normal', False,
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
482 _('convert from a largefiles repo to a normal repo')),
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
483 ],
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
484 _('hg lfconvert SOURCE DEST [FILE ...]')),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
485 }