Mercurial > public > mercurial-scm > hg

annotate hgext/largefiles/lfcommands.py @ 23189:fb139f5553d6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
largefiles: get function to write status messages via "getstatuswriter()" This patch makes "updatelfiles()" get appropriate function to write largefiles specific status messages via "getstatuswriter()". This patch introduces None as "print messages if needed", because True (forcibly writing) and False (forcibly ignoring) are already used for "printmessage" of "updatelfiles". Subsequent patch will move "avoid printing messages only while automated committing" decision from caller of "updatelfiles()" into "getstatuswriter()".
author FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
date Wed, 05 Nov 2014 23:24:47 +0900
parents e53f6b72a0e4
children 4be754832829
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
1 # Copyright 2009-2010 Gregory P. Ward
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
3 # Copyright 2010-2011 Fog Creek Software
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
4 # Copyright 2010-2011 Unity Technologies
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
5 #
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
8
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
9 '''High-level command function for lfconvert, plus the cmdtable.'''
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
10
18728
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
11 import os, errno
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
12 import shutil
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
13
16691
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
14 from mercurial import util, match as match_, hg, node, context, error, \
17773
434e5bd615fc commands: don't infer repo for commands like update (issue2748)
Siddharth Agarwal <sid0@fb.com>
parents: 17537
diff changeset
15 cmdutil, scmutil, commands
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
16 from mercurial.i18n import _
16717
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
17 from mercurial.lock import release
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
18
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
19 import lfutil
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
20 import basestore
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
21
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
22 # -- Commands ----------------------------------------------------------
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
23
21242
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
24 cmdtable = {}
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
25 command = cmdutil.command(cmdtable)
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
26
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
27 @command('lfconvert',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
28 [('s', 'size', '',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
29 _('minimum size (MB) for files to be converted as largefiles'), 'SIZE'),
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
30 ('', 'to-normal', False,
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
31 _('convert from a largefiles repo to a normal repo')),
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
32 ],
21770
15d434bee41c largefiles: define norepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21689
diff changeset
33 _('hg lfconvert SOURCE DEST [FILE ...]'),
21785
a730b002c5db largefiles: define inferrepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21770
diff changeset
34 norepo=True,
a730b002c5db largefiles: define inferrepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21770
diff changeset
35 inferrepo=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
36 def lfconvert(ui, src, dest, *pats, **opts):
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
37 '''convert a normal repository to a largefiles repository
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
38
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
39 Convert repository SOURCE to a new repository DEST, identical to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
40 SOURCE except that certain files will be converted as largefiles:
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
41 specifically, any file that matches any PATTERN *or* whose size is
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
42 above the minimum size threshold is converted as a largefile. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
43 size used to determine whether or not to track a file as a
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
44 largefile is the size of the first version of the file. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
45 minimum size can be specified either with --size or in
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
46 configuration as ``largefiles.size``.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
47
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
48 After running this command you will need to make sure that
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
49 largefiles is enabled anywhere you intend to push the new
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
50 repository.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
51
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
52 Use --to-normal to convert largefiles back to normal files; after
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
53 this, the DEST repository can be used without largefiles at all.'''
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
54
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
55 if opts['to_normal']:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
56 tolfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
57 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
58 tolfile = True
15227
a7686abf73a6 largefiles: factor out lfutil.getminsize()
Greg Ward <greg@gerg.ca>
parents: 15224
diff changeset
59 size = lfutil.getminsize(ui, True, opts.get('size'), default=None)
15340
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
60
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
61 if not hg.islocal(src):
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
62 raise util.Abort(_('%s is not a local Mercurial repo') % src)
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
63 if not hg.islocal(dest):
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
64 raise util.Abort(_('%s is not a local Mercurial repo') % dest)
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
65
15339
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
66 rsrc = hg.repository(ui, src)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
67 ui.status(_('initializing destination %s\n') % dest)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
68 rdst = hg.repository(ui, dest, create=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
69
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
70 success = False
16717
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
71 dstwlock = dstlock = None
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
72 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
73 # Lock destination to prevent modification while it is converted to.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
74 # Don't need to lock src because we are just reading from its history
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
75 # which can't change.
16717
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
76 dstwlock = rdst.wlock()
16247
d87d9d8a8e03 largefiles: remove use of underscores that breaks coding convention
Na'Tosha Bard <natosha@unity3d.com>
parents: 16231
diff changeset
77 dstlock = rdst.lock()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
78
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
79 # Get a list of all changesets in the source. The easy way to do this
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17299
diff changeset
80 # is to simply walk the changelog, using changelog.nodesbetween().
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
81 # Take a look at mercurial/revlog.py:639 for more details.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
82 # Use a generator instead of a list to decrease memory usage
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
83 ctxs = (rsrc[ctx] for ctx in rsrc.changelog.nodesbetween(None,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
84 rsrc.heads())[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
85 revmap = {node.nullid: node.nullid}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
86 if tolfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
87 lfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
88 normalfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
89 if not pats:
15579
6c5e6ebe0812 largefiles: use "ui.configlist()" to get largefiles.patterns configuration
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15472
diff changeset
90 pats = ui.configlist(lfutil.longname, 'patterns', default=[])
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
91 if pats:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
92 matcher = match_.match(rsrc.root, '', list(pats))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
93 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
94 matcher = None
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
95
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
96 lfiletohash = {}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
97 for ctx in ctxs:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
98 ui.progress(_('converting revisions'), ctx.rev(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
99 unit=_('revision'), total=rsrc['tip'].rev())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
100 _lfconvert_addchangeset(rsrc, rdst, ctx, revmap,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
101 lfiles, normalfiles, matcher, size, lfiletohash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
102 ui.progress(_('converting revisions'), None)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
103
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
104 if os.path.exists(rdst.wjoin(lfutil.shortname)):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
105 shutil.rmtree(rdst.wjoin(lfutil.shortname))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
106
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
107 for f in lfiletohash.keys():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
108 if os.path.isfile(rdst.wjoin(f)):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
109 os.unlink(rdst.wjoin(f))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
110 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
111 os.removedirs(os.path.dirname(rdst.wjoin(f)))
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
112 except OSError:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
113 pass
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
114
15303
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
115 # If there were any files converted to largefiles, add largefiles
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
116 # to the destination repository's requirements.
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
117 if lfiles:
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
118 rdst.requirements.add('largefiles')
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
119 rdst._writerequirements()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
120 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
121 for ctx in ctxs:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
122 ui.progress(_('converting revisions'), ctx.rev(),
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
123 unit=_('revision'), total=rsrc['tip'].rev())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
124 _addchangeset(ui, rsrc, rdst, ctx, revmap)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
125
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
126 ui.progress(_('converting revisions'), None)
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
127 success = True
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
128 finally:
16717
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
129 rdst.dirstate.clear()
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
130 release(dstlock, dstwlock)
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
131 if not success:
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
132 # we failed, remove the new directory
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
133 shutil.rmtree(rdst.root)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
134
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
135 def _addchangeset(ui, rsrc, rdst, ctx, revmap):
17299
e51d4aedace9 check-code: indent 4 spaces in py files
Mads Kiilerich <mads@kiilerich.com>
parents: 17155
diff changeset
136 # Convert src parents to dst parents
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
137 parents = _convertparents(ctx, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
138
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
139 # Generate list of changed files
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
140 files = _getchangedfiles(ctx, parents)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
141
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
142 def getfilectx(repo, memctx, f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
143 if lfutil.standin(f) in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
144 # if the file isn't in the manifest then it was removed
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
145 # or renamed, raise IOError to indicate this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
146 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
147 fctx = ctx.filectx(lfutil.standin(f))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
148 except error.LookupError:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22197
diff changeset
149 return None
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
150 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
151 if renamed:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
152 renamed = lfutil.splitstandin(renamed[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
153
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
154 hash = fctx.data().strip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
155 path = lfutil.findfile(rsrc, hash)
17823
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
156
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
157 # If one file is missing, likely all files from this rev are
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
158 if path is None:
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
159 cachelfiles(ui, rsrc, ctx.node())
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
160 path = lfutil.findfile(rsrc, hash)
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
161
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
162 if path is None:
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
163 raise util.Abort(
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
164 _("missing largefile \'%s\' from revision %s")
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
165 % (f, node.hex(ctx.node())))
0fc1ce271ee6 largefiles: fix a traceback in lfconvert if a largefile is missing (issue3519)
Matt Harbison <matt_harbison@yahoo.com>
parents: 17773
diff changeset
166
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
167 data = ''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
168 fd = None
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
169 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
170 fd = open(path, 'rb')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
171 data = fd.read()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
172 finally:
15172
fb1dcd2aae2a largefiles: fix multistatement line
Matt Mackall <mpm@selenic.com>
parents: 15171
diff changeset
173 if fd:
fb1dcd2aae2a largefiles: fix multistatement line
Matt Mackall <mpm@selenic.com>
parents: 15171
diff changeset
174 fd.close()
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
175 return context.memfilectx(repo, f, data, 'l' in fctx.flags(),
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
176 'x' in fctx.flags(), renamed)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
177 else:
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
178 return _getnormalcontext(repo, ctx, f, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
179
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
180 dstfiles = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
181 for file in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
182 if lfutil.isstandin(file):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
183 dstfiles.append(lfutil.splitstandin(file))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
184 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
185 dstfiles.append(file)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
186 # Commit
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
187 _commitcontext(rdst, parents, ctx, dstfiles, getfilectx, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
188
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
189 def _lfconvert_addchangeset(rsrc, rdst, ctx, revmap, lfiles, normalfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
190 matcher, size, lfiletohash):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
191 # Convert src parents to dst parents
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
192 parents = _convertparents(ctx, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
193
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
194 # Generate list of changed files
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
195 files = _getchangedfiles(ctx, parents)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
196
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
197 dstfiles = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
198 for f in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
199 if f not in lfiles and f not in normalfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
200 islfile = _islfile(f, ctx, matcher, size)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
201 # If this file was renamed or copied then copy
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17299
diff changeset
202 # the largefile-ness of its predecessor
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
203 if f in ctx.manifest():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
204 fctx = ctx.filectx(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
205 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
206 renamedlfile = renamed and renamed[0] in lfiles
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
207 islfile |= renamedlfile
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
208 if 'l' in fctx.flags():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
209 if renamedlfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
210 raise util.Abort(
15380
a53888685a6c largefiles: fix uppercase in abort message
Martin Geisler <mg@aragost.com>
parents: 15371
diff changeset
211 _('renamed/copied largefile %s becomes symlink')
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
212 % f)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
213 islfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
214 if islfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
215 lfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
216 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
217 normalfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
218
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
219 if f in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
220 dstfiles.append(lfutil.standin(f))
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
221 # largefile in manifest if it has not been removed/renamed
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
222 if f in ctx.manifest():
15808
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
223 fctx = ctx.filectx(f)
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
224 if 'l' in fctx.flags():
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
225 renamed = fctx.renamed()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
226 if renamed and renamed[0] in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
227 raise util.Abort(_('largefile %s becomes symlink') % f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
228
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
229 # largefile was modified, update standins
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
230 m = util.sha1('')
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
231 m.update(ctx[f].data())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
232 hash = m.hexdigest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
233 if f not in lfiletohash or lfiletohash[f] != hash:
19089
0509ae083ec1 largefiles: use repo.wwrite for writing standins (issue3909)
Mads Kiilerich <madski@unity3d.com>
parents: 18976
diff changeset
234 rdst.wwrite(f, ctx[f].data(), ctx[f].flags())
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
235 executable = 'x' in ctx[f].flags()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
236 lfutil.writestandin(rdst, lfutil.standin(f), hash,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
237 executable)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
238 lfiletohash[f] = hash
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
239 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
240 # normal file
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
241 dstfiles.append(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
242
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
243 def getfilectx(repo, memctx, f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
244 if lfutil.isstandin(f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
245 # if the file isn't in the manifest then it was removed
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
246 # or renamed, raise IOError to indicate this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
247 srcfname = lfutil.splitstandin(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
248 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
249 fctx = ctx.filectx(srcfname)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
250 except error.LookupError:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22197
diff changeset
251 return None
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
252 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
253 if renamed:
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
254 # standin is always a largefile because largefile-ness
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
255 # doesn't change after rename or copy
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
256 renamed = lfutil.standin(renamed[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
257
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
258 return context.memfilectx(repo, f, lfiletohash[srcfname] + '\n',
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
259 'l' in fctx.flags(), 'x' in fctx.flags(),
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
260 renamed)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
261 else:
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
262 return _getnormalcontext(repo, ctx, f, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
263
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
264 # Commit
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
265 _commitcontext(rdst, parents, ctx, dstfiles, getfilectx, revmap)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
266
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
267 def _commitcontext(rdst, parents, ctx, dstfiles, getfilectx, revmap):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
268 mctx = context.memctx(rdst, parents, ctx.description(), dstfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
269 getfilectx, ctx.user(), ctx.date(), ctx.extra())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
270 ret = rdst.commitctx(mctx)
16551
ebf6d38c9063 localrepo: add setparents() to adjust dirstate copies (issue3407)
Patrick Mezard <patrick@mezard.eu>
parents: 16439
diff changeset
271 rdst.setparents(ret)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
272 revmap[ctx.node()] = rdst.changelog.tip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
273
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
274 # Generate list of changed files
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
275 def _getchangedfiles(ctx, parents):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
276 files = set(ctx.files())
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
277 if node.nullid not in parents:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
278 mc = ctx.manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
279 mp1 = ctx.parents()[0].manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
280 mp2 = ctx.parents()[1].manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
281 files |= (set(mp1) | set(mp2)) - set(mc)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
282 for f in mc:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
283 if mc[f] != mp1.get(f, None) or mc[f] != mp2.get(f, None):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
284 files.add(f)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
285 return files
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
286
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
287 # Convert src parents to dst parents
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
288 def _convertparents(ctx, revmap):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
289 parents = []
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
290 for p in ctx.parents():
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
291 parents.append(revmap[p.node()])
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
292 while len(parents) < 2:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
293 parents.append(node.nullid)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
294 return parents
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
295
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
296 # Get memfilectx for a normal file
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
297 def _getnormalcontext(repo, ctx, f, revmap):
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
298 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
299 fctx = ctx.filectx(f)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
300 except error.LookupError:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22197
diff changeset
301 return None
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
302 renamed = fctx.renamed()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
303 if renamed:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
304 renamed = renamed[0]
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
305
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
306 data = fctx.data()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
307 if f == '.hgtags':
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
308 data = _converttags (repo.ui, revmap, data)
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
309 return context.memfilectx(repo, f, data, 'l' in fctx.flags(),
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
310 'x' in fctx.flags(), renamed)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
311
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
312 # Remap tag data using a revision map
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
313 def _converttags(ui, revmap, data):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
314 newdata = []
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
315 for line in data.splitlines():
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
316 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
317 id, name = line.split(' ', 1)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
318 except ValueError:
20868
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
319 ui.warn(_('skipping incorrectly formatted tag %s\n')
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
320 % line)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
321 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
322 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
323 newid = node.bin(id)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
324 except TypeError:
20868
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
325 ui.warn(_('skipping incorrectly formatted id %s\n')
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
326 % id)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
327 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
328 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
329 newdata.append('%s %s\n' % (node.hex(revmap[newid]),
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
330 name))
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
331 except KeyError:
16231
ce292f1379ba i18n: fix all remaining uses of % inside _()
Matt Mackall <mpm@selenic.com>
parents: 15909
diff changeset
332 ui.warn(_('no mapping for id %s\n') % id)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
333 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
334 return ''.join(newdata)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
335
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
336 def _islfile(file, ctx, matcher, size):
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
337 '''Return true if file should be considered a largefile, i.e.
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
338 matcher matches it or it is larger than size.'''
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
339 # never store special .hg* files as largefiles
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
340 if file == '.hgtags' or file == '.hgignore' or file == '.hgsigs':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
341 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
342 if matcher and matcher(file):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
343 return True
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
344 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
345 return ctx.filectx(file).size() >= size * 1024 * 1024
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
346 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
347 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
348
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
349 def uploadlfiles(ui, rsrc, rdst, files):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
350 '''upload largefiles to the central store'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
351
15317
41f371150ccb largefiles: make the store primary, and the user cache secondary
Benjamin Pollack <benjamin@bitquabit.com>
parents: 15313
diff changeset
352 if not files:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
353 return
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
354
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
355 store = basestore._openstore(rsrc, rdst, put=True)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
356
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
357 at = 0
17127
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
358 ui.debug("sending statlfile command for %d largefiles\n" % len(files))
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
359 retval = store.exists(files)
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
360 files = filter(lambda h: not retval[h], files)
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
361 ui.debug("%d largefiles need to be uploaded\n" % len(files))
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
362
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
363 for hash in files:
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
364 ui.progress(_('uploading largefiles'), at, unit='largefile',
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
365 total=len(files))
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
366 source = lfutil.findfile(rsrc, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
367 if not source:
15253
67d010779907 largefiles: improve error reporting
Greg Ward <greg@gerg.ca>
parents: 15252
diff changeset
368 raise util.Abort(_('largefile %s missing from store'
67d010779907 largefiles: improve error reporting
Greg Ward <greg@gerg.ca>
parents: 15252
diff changeset
369 ' (needs to be uploaded)') % hash)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
370 # XXX check for errors here
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
371 store.put(source, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
372 at += 1
15173
3d27a8ff895f largefiles: mark a string for translation
Matt Mackall <mpm@selenic.com>
parents: 15172
diff changeset
373 ui.progress(_('uploading largefiles'), None)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
374
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
375 def verifylfiles(ui, repo, all=False, contents=False):
18574
4db9e31ae605 largefiles: docstrings for verify methods
Mads Kiilerich <mads@kiilerich.com>
parents: 18294
diff changeset
376 '''Verify that every largefile revision in the current changeset
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
377 exists in the central store. With --contents, also verify that
18574
4db9e31ae605 largefiles: docstrings for verify methods
Mads Kiilerich <mads@kiilerich.com>
parents: 18294
diff changeset
378 the contents of each local largefile file revision are correct (SHA-1 hash
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
379 matches the revision ID). With --all, check every changeset in
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
380 this repository.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
381 if all:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
382 # Pass a list to the function rather than an iterator because we know a
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
383 # list will work.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
384 revs = range(len(repo))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
385 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
386 revs = ['.']
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
387
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
388 store = basestore._openstore(repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
389 return store.verify(revs, contents=contents)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
390
16700
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
391 def cachelfiles(ui, repo, node, filelist=None):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
392 '''cachelfiles ensures that all largefiles needed by the specified revision
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
393 are present in the repository's largefile cache.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
394
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
395 returns a tuple (cached, missing). cached is the list of files downloaded
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
396 by this operation; missing is the list of files that were needed but could
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
397 not be found.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
398 lfiles = lfutil.listlfiles(repo, node)
16700
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
399 if filelist:
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
400 lfiles = set(lfiles) & set(filelist)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
401 toget = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
402
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
403 for lfile in lfiles:
18728
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
404 try:
15860
3ecce805ac13 largefiles: correctly download new largefiles when merging
Na'Tosha Bard <natosha@unity3d.com>
parents: 15811
diff changeset
405 expectedhash = repo[node][lfutil.standin(lfile)].data().strip()
18728
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
406 except IOError, err:
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
407 if err.errno == errno.ENOENT:
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
408 continue # node must be None and standin wasn't found in wctx
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
409 raise
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
410 if not lfutil.findfile(repo, expectedhash):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
411 toget.append((lfile, expectedhash))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
412
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
413 if toget:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
414 store = basestore._openstore(repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
415 ret = store.get(toget)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
416 return ret
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
417
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
418 return ([], [])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
419
16691
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
420 def downloadlfiles(ui, repo, rev=None):
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
421 matchfn = scmutil.match(repo[None],
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
422 [repo.wjoin(lfutil.shortname)], {})
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
423 def prepare(ctx, fns):
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
424 pass
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
425 totalsuccess = 0
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
426 totalmissing = 0
18722
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
427 if rev != []: # walkchangerevs on empty list would return all revs
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
428 for ctx in cmdutil.walkchangerevs(repo, matchfn, {'rev' : rev},
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
429 prepare):
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
430 success, missing = cachelfiles(ui, repo, ctx.node())
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
431 totalsuccess += len(success)
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
432 totalmissing += len(missing)
16691
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
433 ui.status(_("%d additional largefiles cached\n") % totalsuccess)
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
434 if totalmissing > 0:
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
435 ui.status(_("%d largefiles failed to download\n") % totalmissing)
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
436 return totalsuccess, totalmissing
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
437
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
438 def updatelfiles(ui, repo, filelist=None, printmessage=None,
21934
0cb34b3991f8 largefiles: use "normallookup" on "lfdirstate" while reverting
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 21933
diff changeset
439 normallookup=False):
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
440 '''Update largefiles according to standins in the working directory
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
441
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
442 If ``printmessage`` is other than ``None``, it means "print (or
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
443 ignore, for false) message forcibly".
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
444 '''
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
445 statuswriter = lfutil.getstatuswriter(ui, repo, printmessage)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
446 wlock = repo.wlock()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
447 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
448 lfdirstate = lfutil.openlfdirstate(ui, repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
449 lfiles = set(lfutil.listlfiles(repo)) | set(lfdirstate)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
450
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
451 if filelist is not None:
22197
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
452 filelist = set(filelist)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
453 lfiles = [f for f in lfiles if f in filelist]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
454
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
455 update = {}
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
456 updated, removed = 0, 0
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
457 for lfile in lfiles:
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
458 abslfile = repo.wjoin(lfile)
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
459 absstandin = repo.wjoin(lfutil.standin(lfile))
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
460 if os.path.exists(absstandin):
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
461 if (os.path.exists(absstandin + '.orig') and
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
462 os.path.exists(abslfile)):
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
463 shutil.copyfile(abslfile, abslfile + '.orig')
21083
20b0c49c032c largefiles: update should only create a .orig backup of a largefile once
Mads Kiilerich <madski@unity3d.com>
parents: 20868
diff changeset
464 util.unlinkpath(absstandin + '.orig')
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
465 expecthash = lfutil.readstandin(repo, lfile)
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
466 if (expecthash != '' and
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
467 (not os.path.exists(abslfile) or
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
468 expecthash != lfutil.hashfile(abslfile))):
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
469 if lfile not in repo[None]: # not switched to normal file
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
470 util.unlinkpath(abslfile, ignoremissing=True)
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
471 # use normallookup() to allocate an entry in largefiles
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
472 # dirstate, because lack of it misleads
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
473 # lfilesrepo.status() into recognition that such cache
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
474 # missing files are removed.
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
475 lfdirstate.normallookup(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
476 update[lfile] = expecthash
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
477 else:
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
478 # Remove lfiles for which the standin is deleted, unless the
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
479 # lfile is added to the repository again. This happens when a
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
480 # largefile is converted back to a normal file: the standin
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
481 # disappears, but a new (normal) file appears as the lfile.
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
482 if (os.path.exists(abslfile) and
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
483 repo.dirstate.normalize(lfile) not in repo[None]):
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
484 util.unlinkpath(abslfile)
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
485 removed += 1
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
486
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
487 # largefile processing might be slow and be interrupted - be prepared
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
488 lfdirstate.write()
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
489
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
490 if lfiles:
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
491 statuswriter(_('getting changed largefiles\n'))
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
492 cachelfiles(ui, repo, None, lfiles)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
493
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
494 for lfile in lfiles:
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
495 update1 = 0
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
496
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
497 expecthash = update.get(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
498 if expecthash:
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
499 if not lfutil.copyfromcache(repo, expecthash, lfile):
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
500 # failed ... but already removed and set to normallookup
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
501 continue
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
502 # Synchronize largefile dirstate to the last modified
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
503 # time of the file
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
504 lfdirstate.normal(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
505 update1 = 1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
506
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
507 # copy the state of largefile standin from the repository's
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
508 # dirstate to its state in the lfdirstate.
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
509 abslfile = repo.wjoin(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
510 absstandin = repo.wjoin(lfutil.standin(lfile))
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
511 if os.path.exists(absstandin):
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
512 mode = os.stat(absstandin).st_mode
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
513 if mode != os.stat(abslfile).st_mode:
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
514 os.chmod(abslfile, mode)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
515 update1 = 1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
516
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
517 updated += update1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
518
22095
cb62d77c7a01 largefiles: factor out synchronization of lfdirstate for future use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 21934
diff changeset
519 lfutil.synclfdirstate(repo, lfdirstate, lfile, normallookup)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
520
22197
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
521 if filelist is not None:
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
522 # If "local largefile" is chosen at file merging, it is
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
523 # not listed in "filelist" (= dirstate syncing is
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
524 # omitted), because the standin file is not changed before and
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
525 # after merging.
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
526 # But the status of such files may have to be changed by
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
527 # merging. For example, locally modified ("M") largefile
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
528 # has to become re-added("A"), if it is "normal" file in
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
529 # the target revision of linear-merging.
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
530 for lfile in lfdirstate:
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
531 if lfile not in filelist:
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
532 lfutil.synclfdirstate(repo, lfdirstate, lfile, True)
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
533
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
534 lfdirstate.write()
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
535 if lfiles:
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
536 statuswriter(_('%d largefiles updated, %d removed\n') % (updated,
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
537 removed))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
538 finally:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
539 wlock.release()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
540
21242
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
541 @command('lfpull',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
542 [('r', 'rev', [], _('pull largefiles for these revisions'))
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
543 ] + commands.remoteopts,
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
544 _('-r REV... [-e CMD] [--remotecmd CMD] [SOURCE]'))
18976
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
545 def lfpull(ui, repo, source="default", **opts):
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
546 """pull largefiles for the specified revisions from the specified source
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
547
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
548 Pull largefiles that are referenced from local changesets but missing
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
549 locally, pulling from a remote repository to the local cache.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
550
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
551 If SOURCE is omitted, the 'default' path will be used.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
552 See :hg:`help urls` for more information.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
553
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
554 .. container:: verbose
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
555
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
556 Some examples:
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
557
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
558 - pull largefiles for all branch heads::
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
559
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
560 hg lfpull -r "head() and not closed()"
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
561
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
562 - pull largefiles on the default branch::
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
563
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
564 hg lfpull -r "branch(default)"
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
565 """
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
566 repo.lfpullsource = source
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
567
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
568 revs = opts.get('rev', [])
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
569 if not revs:
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
570 raise util.Abort(_('no revisions specified'))
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
571 revs = scmutil.revrange(repo, revs)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
572
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
573 numcached = 0
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
574 for rev in revs:
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
575 ui.note(_('pulling largefiles for revision %s\n') % rev)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
576 (cached, missing) = cachelfiles(ui, repo, rev)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
577 numcached += len(cached)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
578 ui.status(_("%d largefiles cached\n") % numcached)