Mercurial > public > mercurial-scm > hg

annotate hgext/largefiles/lfcommands.py @ 29341:0d83ad967bf8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
cleanup: replace uses of util.(md5|sha1|sha256|sha512) with hashlib.\1 All versions of Python we support or hope to support make the hash functions available in the same way under the same name, so we may as well drop the util forwards.
author Augie Fackler <raf@durin42.com>
date Fri, 10 Jun 2016 00:12:33 -0400
parents 5ec25534ef4f
children 85868ecf2c0d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
1 # Copyright 2009-2010 Gregory P. Ward
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
3 # Copyright 2010-2011 Fog Creek Software
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
4 # Copyright 2010-2011 Unity Technologies
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
5 #
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
8
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
9 '''High-level command function for lfconvert, plus the cmdtable.'''
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
10 from __future__ import absolute_import
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
11
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
12 import errno
29341
0d83ad967bf8 cleanup: replace uses of util.(md5|sha1|sha256|sha512) with hashlib.\1
Augie Fackler <raf@durin42.com>
parents: 29317
diff changeset
13 import hashlib
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
14 import os
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
15 import shutil
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
16
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
17 from mercurial.i18n import _
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
18
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
19 from mercurial import (
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
20 cmdutil,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
21 commands,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
22 context,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
23 error,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
24 hg,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
25 lock,
29317
5ec25534ef4f largefiles: rename match_ to matchmod import in lfcommands
liscju <piotr.listkiewicz@gmail.com>
parents: 29308
diff changeset
26 match as matchmod,
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
27 node,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
28 scmutil,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
29 util,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
30 )
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
31
29308
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
32 from ..convert import (
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
33 convcmd,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
34 filemap,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
35 )
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
36
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
37 from . import (
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
38 lfutil,
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
39 storefactory
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
40 )
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
41
8c378a7d2aa6 py3: make largefiles/lfcommands.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 29305
diff changeset
42 release = lock.release
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
43
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
44 # -- Commands ----------------------------------------------------------
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
45
21242
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
46 cmdtable = {}
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
47 command = cmdutil.command(cmdtable)
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
48
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
49 @command('lfconvert',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
50 [('s', 'size', '',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
51 _('minimum size (MB) for files to be converted as largefiles'), 'SIZE'),
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
52 ('', 'to-normal', False,
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
53 _('convert from a largefiles repo to a normal repo')),
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
54 ],
21770
15d434bee41c largefiles: define norepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21689
diff changeset
55 _('hg lfconvert SOURCE DEST [FILE ...]'),
21785
a730b002c5db largefiles: define inferrepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21770
diff changeset
56 norepo=True,
a730b002c5db largefiles: define inferrepo in command decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21770
diff changeset
57 inferrepo=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
58 def lfconvert(ui, src, dest, *pats, **opts):
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
59 '''convert a normal repository to a largefiles repository
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
60
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
61 Convert repository SOURCE to a new repository DEST, identical to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
62 SOURCE except that certain files will be converted as largefiles:
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
63 specifically, any file that matches any PATTERN *or* whose size is
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
64 above the minimum size threshold is converted as a largefile. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
65 size used to determine whether or not to track a file as a
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
66 largefile is the size of the first version of the file. The
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
67 minimum size can be specified either with --size or in
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
68 configuration as ``largefiles.size``.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
69
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
70 After running this command you will need to make sure that
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
71 largefiles is enabled anywhere you intend to push the new
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
72 repository.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
73
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
74 Use --to-normal to convert largefiles back to normal files; after
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15228
diff changeset
75 this, the DEST repository can be used without largefiles at all.'''
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
76
15332
0db47b8d025f largefiles: rename lfconvert --tonormal option to --to-normal
Greg Ward <greg@gerg.ca>
parents: 15317
diff changeset
77 if opts['to_normal']:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
78 tolfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
79 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
80 tolfile = True
15227
a7686abf73a6 largefiles: factor out lfutil.getminsize()
Greg Ward <greg@gerg.ca>
parents: 15224
diff changeset
81 size = lfutil.getminsize(ui, True, opts.get('size'), default=None)
15340
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
82
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
83 if not hg.islocal(src):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
84 raise error.Abort(_('%s is not a local Mercurial repo') % src)
15340
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
85 if not hg.islocal(dest):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
86 raise error.Abort(_('%s is not a local Mercurial repo') % dest)
15340
0e58513cc59a largefiles: rearrange how lfconvert detects non-local repos
Greg Ward <greg@gerg.ca>
parents: 15339
diff changeset
87
15339
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
88 rsrc = hg.repository(ui, src)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
89 ui.status(_('initializing destination %s\n') % dest)
be1377d19018 largefiles: test lfconvert error handling; remove redundant code
Greg Ward <greg@gerg.ca>
parents: 15332
diff changeset
90 rdst = hg.repository(ui, dest, create=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
91
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
92 success = False
16717
1eede2ea2041 largefiles: use wlock for lfconvert (issue3444)
Mads Kiilerich <mads@kiilerich.com>
parents: 16551
diff changeset
93 dstwlock = dstlock = None
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
94 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
95 # Get a list of all changesets in the source. The easy way to do this
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17299
diff changeset
96 # is to simply walk the changelog, using changelog.nodesbetween().
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
97 # Take a look at mercurial/revlog.py:639 for more details.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
98 # Use a generator instead of a list to decrease memory usage
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
99 ctxs = (rsrc[ctx] for ctx in rsrc.changelog.nodesbetween(None,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
100 rsrc.heads())[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
101 revmap = {node.nullid: node.nullid}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
102 if tolfile:
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
103 # Lock destination to prevent modification while it is converted to.
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
104 # Don't need to lock src because we are just reading from its
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
105 # history which can't change.
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
106 dstwlock = rdst.wlock()
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
107 dstlock = rdst.lock()
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
108
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
109 lfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
110 normalfiles = set()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
111 if not pats:
15579
6c5e6ebe0812 largefiles: use "ui.configlist()" to get largefiles.patterns configuration
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15472
diff changeset
112 pats = ui.configlist(lfutil.longname, 'patterns', default=[])
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
113 if pats:
29317
5ec25534ef4f largefiles: rename match_ to matchmod import in lfcommands
liscju <piotr.listkiewicz@gmail.com>
parents: 29308
diff changeset
114 matcher = matchmod.match(rsrc.root, '', list(pats))
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
115 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
116 matcher = None
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
117
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
118 lfiletohash = {}
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
119 for ctx in ctxs:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
120 ui.progress(_('converting revisions'), ctx.rev(),
28464
6e34690230c0 largefiles: use revisions as a ui.progress unit
Anton Shestakov <av6@dwimlabs.net>
parents: 28463
diff changeset
121 unit=_('revisions'), total=rsrc['tip'].rev())
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
122 _lfconvert_addchangeset(rsrc, rdst, ctx, revmap,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
123 lfiles, normalfiles, matcher, size, lfiletohash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
124 ui.progress(_('converting revisions'), None)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
125
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
126 if rdst.wvfs.exists(lfutil.shortname):
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
127 rdst.wvfs.rmtree(lfutil.shortname)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
128
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
129 for f in lfiletohash.keys():
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
130 if rdst.wvfs.isfile(f):
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
131 rdst.wvfs.unlink(f)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
132 try:
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
133 rdst.wvfs.removedirs(rdst.wvfs.dirname(f))
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
134 except OSError:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
135 pass
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
136
15303
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
137 # If there were any files converted to largefiles, add largefiles
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
138 # to the destination repository's requirements.
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
139 if lfiles:
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
140 rdst.requirements.add('largefiles')
07811b3b119b largefiles: include 'largefiles' in converted repository requirements
Eli Carter <eli.carter@tektronix.com>
parents: 15255
diff changeset
141 rdst._writerequirements()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
142 else:
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
143 class lfsource(filemap.filemap_source):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
144 def __init__(self, ui, source):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
145 super(lfsource, self).__init__(ui, source, None)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
146 self.filemapper.rename[lfutil.shortname] = '.'
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
147
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
148 def getfile(self, name, rev):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
149 realname, realrev = rev
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
150 f = super(lfsource, self).getfile(name, rev)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
151
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
152 if (not realname.startswith(lfutil.shortnameslash)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
153 or f[0] is None):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
154 return f
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
155
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
156 # Substitute in the largefile data for the hash
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
157 hash = f[0].strip()
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
158 path = lfutil.findfile(rsrc, hash)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
159
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
160 if path is None:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
161 raise error.Abort(_("missing largefile for '%s' in %s")
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
162 % (realname, realrev))
27774
8ceaaf63ca80 largefiles: use util.readfile in lfconvert
Bryan O'Sullivan <bryano@fb.com>
parents: 27651
diff changeset
163 return util.readfile(path), f[1]
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
164
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
165 class converter(convcmd.converter):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
166 def __init__(self, ui, source, dest, revmapfile, opts):
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
167 src = lfsource(ui, source)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
168
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
169 super(converter, self).__init__(ui, src, dest, revmapfile,
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
170 opts)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
171
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
172 found, missing = downloadlfiles(ui, rsrc)
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
173 if missing != 0:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
174 raise error.Abort(_("all largefiles must be present locally"))
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
175
25560
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
176 orig = convcmd.converter
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
177 convcmd.converter = converter
25560
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
178
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
179 try:
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
180 convcmd.convert(ui, src, dest)
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
181 finally:
2b2108c35bfc largefiles: restore the original converter class after lfconvert --to-normal
Matt Harbison <matt_harbison@yahoo.com>
parents: 25508
diff changeset
182 convcmd.converter = orig
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
183 success = True
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
184 finally:
25325
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
185 if tolfile:
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
186 rdst.dirstate.clear()
fcd2f9b06629 largefiles: use the convert extension for 'lfconvert --to-normal'
Matt Harbison <matt_harbison@yahoo.com>
parents: 24788
diff changeset
187 release(dstlock, dstwlock)
15171
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
188 if not success:
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
189 # we failed, remove the new directory
547da6115d1d largefiles: eliminate naked exceptions
Matt Mackall <mpm@selenic.com>
parents: 15170
diff changeset
190 shutil.rmtree(rdst.root)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
191
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
192 def _lfconvert_addchangeset(rsrc, rdst, ctx, revmap, lfiles, normalfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
193 matcher, size, lfiletohash):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
194 # Convert src parents to dst parents
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
195 parents = _convertparents(ctx, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
196
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
197 # Generate list of changed files
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
198 files = _getchangedfiles(ctx, parents)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
199
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
200 dstfiles = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
201 for f in files:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
202 if f not in lfiles and f not in normalfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
203 islfile = _islfile(f, ctx, matcher, size)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
204 # If this file was renamed or copied then copy
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17299
diff changeset
205 # the largefile-ness of its predecessor
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
206 if f in ctx.manifest():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
207 fctx = ctx.filectx(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
208 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
209 renamedlfile = renamed and renamed[0] in lfiles
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
210 islfile |= renamedlfile
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
211 if 'l' in fctx.flags():
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
212 if renamedlfile:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
213 raise error.Abort(
15380
a53888685a6c largefiles: fix uppercase in abort message
Martin Geisler <mg@aragost.com>
parents: 15371
diff changeset
214 _('renamed/copied largefile %s becomes symlink')
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
215 % f)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
216 islfile = False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
217 if islfile:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
218 lfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
219 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
220 normalfiles.add(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
221
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
222 if f in lfiles:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
223 dstfiles.append(lfutil.standin(f))
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
224 # largefile in manifest if it has not been removed/renamed
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
225 if f in ctx.manifest():
15808
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
226 fctx = ctx.filectx(f)
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
227 if 'l' in fctx.flags():
62098aeb1e15 largefiles: don't reference uninitialized variable (issue3092)
Levi Bard <levi@unity3d.com>
parents: 15793
diff changeset
228 renamed = fctx.renamed()
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
229 if renamed and renamed[0] in lfiles:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
230 raise error.Abort(_('largefile %s becomes symlink') % f)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
231
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
232 # largefile was modified, update standins
29341
0d83ad967bf8 cleanup: replace uses of util.(md5|sha1|sha256|sha512) with hashlib.\1
Augie Fackler <raf@durin42.com>
parents: 29317
diff changeset
233 m = hashlib.sha1('')
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
234 m.update(ctx[f].data())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
235 hash = m.hexdigest()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
236 if f not in lfiletohash or lfiletohash[f] != hash:
19089
0509ae083ec1 largefiles: use repo.wwrite for writing standins (issue3909)
Mads Kiilerich <madski@unity3d.com>
parents: 18976
diff changeset
237 rdst.wwrite(f, ctx[f].data(), ctx[f].flags())
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
238 executable = 'x' in ctx[f].flags()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
239 lfutil.writestandin(rdst, lfutil.standin(f), hash,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
240 executable)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
241 lfiletohash[f] = hash
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
242 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
243 # normal file
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
244 dstfiles.append(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
245
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
246 def getfilectx(repo, memctx, f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
247 if lfutil.isstandin(f):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
248 # if the file isn't in the manifest then it was removed
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
249 # or renamed, raise IOError to indicate this
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
250 srcfname = lfutil.splitstandin(f)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
251 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
252 fctx = ctx.filectx(srcfname)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
253 except error.LookupError:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22197
diff changeset
254 return None
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
255 renamed = fctx.renamed()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
256 if renamed:
15254
dd03d3a9f888 largefiles: more work on cleaning up comments
Greg Ward <greg@gerg.ca>
parents: 15253
diff changeset
257 # standin is always a largefile because largefile-ness
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
258 # doesn't change after rename or copy
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
259 renamed = lfutil.standin(renamed[0])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
260
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
261 return context.memfilectx(repo, f, lfiletohash[srcfname] + '\n',
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
262 'l' in fctx.flags(), 'x' in fctx.flags(),
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
263 renamed)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
264 else:
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
265 return _getnormalcontext(repo, ctx, f, revmap)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
266
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
267 # Commit
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
268 _commitcontext(rdst, parents, ctx, dstfiles, getfilectx, revmap)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
269
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
270 def _commitcontext(rdst, parents, ctx, dstfiles, getfilectx, revmap):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
271 mctx = context.memctx(rdst, parents, ctx.description(), dstfiles,
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
272 getfilectx, ctx.user(), ctx.date(), ctx.extra())
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
273 ret = rdst.commitctx(mctx)
23276
4be754832829 largefiles: move "copyalltostore" invocation into "markcommitted"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23189
diff changeset
274 lfutil.copyalltostore(rdst, ret)
16551
ebf6d38c9063 localrepo: add setparents() to adjust dirstate copies (issue3407)
Patrick Mezard <patrick@mezard.eu>
parents: 16439
diff changeset
275 rdst.setparents(ret)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
276 revmap[ctx.node()] = rdst.changelog.tip()
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
277
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
278 # Generate list of changed files
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
279 def _getchangedfiles(ctx, parents):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
280 files = set(ctx.files())
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
281 if node.nullid not in parents:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
282 mc = ctx.manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
283 mp1 = ctx.parents()[0].manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
284 mp2 = ctx.parents()[1].manifest()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
285 files |= (set(mp1) | set(mp2)) - set(mc)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
286 for f in mc:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
287 if mc[f] != mp1.get(f, None) or mc[f] != mp2.get(f, None):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
288 files.add(f)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
289 return files
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
290
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
291 # Convert src parents to dst parents
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
292 def _convertparents(ctx, revmap):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
293 parents = []
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
294 for p in ctx.parents():
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
295 parents.append(revmap[p.node()])
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
296 while len(parents) < 2:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
297 parents.append(node.nullid)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
298 return parents
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
299
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
300 # Get memfilectx for a normal file
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
301 def _getnormalcontext(repo, ctx, f, revmap):
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
302 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
303 fctx = ctx.filectx(f)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
304 except error.LookupError:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22197
diff changeset
305 return None
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
306 renamed = fctx.renamed()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
307 if renamed:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
308 renamed = renamed[0]
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
309
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
310 data = fctx.data()
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
311 if f == '.hgtags':
21689
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
312 data = _converttags (repo.ui, revmap, data)
503bb3af70fe memfilectx: call super.__init__ instead of duplicating code
Sean Farley <sean.michael.farley@gmail.com>
parents: 21242
diff changeset
313 return context.memfilectx(repo, f, data, 'l' in fctx.flags(),
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
314 'x' in fctx.flags(), renamed)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
315
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
316 # Remap tag data using a revision map
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
317 def _converttags(ui, revmap, data):
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
318 newdata = []
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
319 for line in data.splitlines():
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
320 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
321 id, name = line.split(' ', 1)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
322 except ValueError:
20868
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
323 ui.warn(_('skipping incorrectly formatted tag %s\n')
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
324 % line)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
325 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
326 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
327 newid = node.bin(id)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
328 except TypeError:
20868
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
329 ui.warn(_('skipping incorrectly formatted id %s\n')
5db105f216c3 i18n: fix "% inside _()" problems
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20063
diff changeset
330 % id)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
331 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
332 try:
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
333 newdata.append('%s %s\n' % (node.hex(revmap[newid]),
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
334 name))
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
335 except KeyError:
16231
ce292f1379ba i18n: fix all remaining uses of % inside _()
Matt Mackall <mpm@selenic.com>
parents: 15909
diff changeset
336 ui.warn(_('no mapping for id %s\n') % id)
15811
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
337 continue
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
338 return ''.join(newdata)
b9886dde3649 largefiles: remove pasted code
Levi Bard <levi@unity3d.com>
parents: 15809
diff changeset
339
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
340 def _islfile(file, ctx, matcher, size):
15252
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
341 '''Return true if file should be considered a largefile, i.e.
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
342 matcher matches it or it is larger than size.'''
6e809bb4f969 largefiles: improve comments, internal docstrings
Greg Ward <greg@gerg.ca>
parents: 15230
diff changeset
343 # never store special .hg* files as largefiles
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
344 if file == '.hgtags' or file == '.hgignore' or file == '.hgsigs':
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
345 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
346 if matcher and matcher(file):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
347 return True
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
348 try:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
349 return ctx.filectx(file).size() >= size * 1024 * 1024
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
350 except error.LookupError:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
351 return False
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
352
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
353 def uploadlfiles(ui, rsrc, rdst, files):
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
354 '''upload largefiles to the central store'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
355
15317
41f371150ccb largefiles: make the store primary, and the user cache secondary
Benjamin Pollack <benjamin@bitquabit.com>
parents: 15313
diff changeset
356 if not files:
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
357 return
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
358
29305
814076f4ace3 largefiles: move basestore._openstore into new module to remove cycle
liscju <piotr.listkiewicz@gmail.com>
parents: 28559
diff changeset
359 store = storefactory._openstore(rsrc, rdst, put=True)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
360
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
361 at = 0
17127
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
362 ui.debug("sending statlfile command for %d largefiles\n" % len(files))
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
363 retval = store.exists(files)
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
364 files = filter(lambda h: not retval[h], files)
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
365 ui.debug("%d largefiles need to be uploaded\n" % len(files))
9e1616307c4c largefiles: batch statlfile requests when pushing a largefiles repo (issue3386)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16727
diff changeset
366
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
367 for hash in files:
28463
19b4a2087dfc largefiles: specify unit for ui.progress when operating on files
Anton Shestakov <av6@dwimlabs.net>
parents: 27820
diff changeset
368 ui.progress(_('uploading largefiles'), at, unit=_('files'),
15170
c1a4a3220711 largefiles: fix over-long lines
Matt Mackall <mpm@selenic.com>
parents: 15168
diff changeset
369 total=len(files))
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
370 source = lfutil.findfile(rsrc, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
371 if not source:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
372 raise error.Abort(_('largefile %s missing from store'
15253
67d010779907 largefiles: improve error reporting
Greg Ward <greg@gerg.ca>
parents: 15252
diff changeset
373 ' (needs to be uploaded)') % hash)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
374 # XXX check for errors here
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
375 store.put(source, hash)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
376 at += 1
15173
3d27a8ff895f largefiles: mark a string for translation
Matt Mackall <mpm@selenic.com>
parents: 15172
diff changeset
377 ui.progress(_('uploading largefiles'), None)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
378
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
379 def verifylfiles(ui, repo, all=False, contents=False):
18574
4db9e31ae605 largefiles: docstrings for verify methods
Mads Kiilerich <mads@kiilerich.com>
parents: 18294
diff changeset
380 '''Verify that every largefile revision in the current changeset
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
381 exists in the central store. With --contents, also verify that
18574
4db9e31ae605 largefiles: docstrings for verify methods
Mads Kiilerich <mads@kiilerich.com>
parents: 18294
diff changeset
382 the contents of each local largefile file revision are correct (SHA-1 hash
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
383 matches the revision ID). With --all, check every changeset in
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
384 this repository.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
385 if all:
25508
b8fd605b0c88 largefiles: ignore hidden changesets with 'verify --large --lfa'
Matt Harbison <matt_harbison@yahoo.com>
parents: 25326
diff changeset
386 revs = repo.revs('all()')
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
387 else:
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
388 revs = ['.']
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
389
29305
814076f4ace3 largefiles: move basestore._openstore into new module to remove cycle
liscju <piotr.listkiewicz@gmail.com>
parents: 28559
diff changeset
390 store = storefactory._openstore(repo)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
391 return store.verify(revs, contents=contents)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
392
16700
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
393 def cachelfiles(ui, repo, node, filelist=None):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
394 '''cachelfiles ensures that all largefiles needed by the specified revision
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
395 are present in the repository's largefile cache.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
396
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
397 returns a tuple (cached, missing). cached is the list of files downloaded
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
398 by this operation; missing is the list of files that were needed but could
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
399 not be found.'''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
400 lfiles = lfutil.listlfiles(repo, node)
16700
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
401 if filelist:
28001e8a5149 largefiles: optimize performance when updating (issue3440)
Na'Tosha Bard <natosha@unity3d.com>
parents: 16691
diff changeset
402 lfiles = set(lfiles) & set(filelist)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
403 toget = []
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
404
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
405 for lfile in lfiles:
18728
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
406 try:
15860
3ecce805ac13 largefiles: correctly download new largefiles when merging
Na'Tosha Bard <natosha@unity3d.com>
parents: 15811
diff changeset
407 expectedhash = repo[node][lfutil.standin(lfile)].data().strip()
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25560
diff changeset
408 except IOError as err:
18728
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
409 if err.errno == errno.ENOENT:
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
410 continue # node must be None and standin wasn't found in wctx
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
411 raise
1e636f7b1cfe largefiles: simplify cachelfiles - don't spend a lot of time checking hashes
Mads Kiilerich <madski@unity3d.com>
parents: 18727
diff changeset
412 if not lfutil.findfile(repo, expectedhash):
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
413 toget.append((lfile, expectedhash))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
414
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
415 if toget:
29305
814076f4ace3 largefiles: move basestore._openstore into new module to remove cycle
liscju <piotr.listkiewicz@gmail.com>
parents: 28559
diff changeset
416 store = storefactory._openstore(repo)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
417 ret = store.get(toget)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
418 return ret
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
419
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
420 return ([], [])
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
421
16691
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
422 def downloadlfiles(ui, repo, rev=None):
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
423 matchfn = scmutil.match(repo[None],
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
424 [repo.wjoin(lfutil.shortname)], {})
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
425 def prepare(ctx, fns):
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
426 pass
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
427 totalsuccess = 0
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
428 totalmissing = 0
18722
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
429 if rev != []: # walkchangerevs on empty list would return all revs
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
430 for ctx in cmdutil.walkchangerevs(repo, matchfn, {'rev' : rev},
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
431 prepare):
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
432 success, missing = cachelfiles(ui, repo, ctx.node())
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
433 totalsuccess += len(success)
f0aa8bbffe60 largefiles: fix download of largefiles from an empty list of changesets
Mads Kiilerich <madski@unity3d.com>
parents: 18294
diff changeset
434 totalmissing += len(missing)
16691
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
435 ui.status(_("%d additional largefiles cached\n") % totalsuccess)
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
436 if totalmissing > 0:
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
437 ui.status(_("%d largefiles failed to download\n") % totalmissing)
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
438 return totalsuccess, totalmissing
7d6a660ca151 largefiles: refactor downloading of all largefiles to generic function
Na'Tosha Bard <natosha@unity3d.com>
parents: 16687
diff changeset
439
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
440 def updatelfiles(ui, repo, filelist=None, printmessage=None,
24788
b8c3a0994b37 largefiles: always consider updatelfiles 'checked' parameter set
Mads Kiilerich <madski@unity3d.com>
parents: 24180
diff changeset
441 normallookup=False):
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
442 '''Update largefiles according to standins in the working directory
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
443
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
444 If ``printmessage`` is other than ``None``, it means "print (or
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
445 ignore, for false) message forcibly".
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
446 '''
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
447 statuswriter = lfutil.getstatuswriter(ui, repo, printmessage)
27820
d2e9cc9edc08 with: use context manager for wlock in updatelfiles
Bryan O'Sullivan <bryano@fb.com>
parents: 27774
diff changeset
448 with repo.wlock():
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
449 lfdirstate = lfutil.openlfdirstate(ui, repo)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
450 lfiles = set(lfutil.listlfiles(repo)) | set(lfdirstate)
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
451
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
452 if filelist is not None:
22197
f72d73937853 largefiles: update lfdirstate for unchanged largefiles during linear merging
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 22095
diff changeset
453 filelist = set(filelist)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
454 lfiles = [f for f in lfiles if f in filelist]
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
455
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
456 update = {}
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
457 updated, removed = 0, 0
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
458 wvfs = repo.wvfs
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
459 for lfile in lfiles:
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
460 rellfile = lfile
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
461 rellfileorig = os.path.relpath(
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
462 scmutil.origpath(ui, repo, wvfs.join(rellfile)),
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
463 start=repo.root)
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
464 relstandin = lfutil.standin(lfile)
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
465 relstandinorig = os.path.relpath(
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
466 scmutil.origpath(ui, repo, wvfs.join(relstandin)),
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
467 start=repo.root)
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
468 if wvfs.exists(relstandin):
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
469 if (wvfs.exists(relstandinorig) and
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
470 wvfs.exists(rellfile)):
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
471 shutil.copyfile(wvfs.join(rellfile),
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
472 wvfs.join(rellfileorig))
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
473 wvfs.unlinkpath(relstandinorig)
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
474 expecthash = lfutil.readstandin(repo, lfile)
24788
b8c3a0994b37 largefiles: always consider updatelfiles 'checked' parameter set
Mads Kiilerich <madski@unity3d.com>
parents: 24180
diff changeset
475 if expecthash != '':
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
476 if lfile not in repo[None]: # not switched to normal file
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
477 wvfs.unlinkpath(rellfile, ignoremissing=True)
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
478 # use normallookup() to allocate an entry in largefiles
24180
d8e0c591781c spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23893
diff changeset
479 # dirstate to prevent lfilesrepo.status() from reporting
d8e0c591781c spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23893
diff changeset
480 # missing files as removed.
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
481 lfdirstate.normallookup(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
482 update[lfile] = expecthash
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
483 else:
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
484 # Remove lfiles for which the standin is deleted, unless the
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
485 # lfile is added to the repository again. This happens when a
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
486 # largefile is converted back to a normal file: the standin
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
487 # disappears, but a new (normal) file appears as the lfile.
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
488 if (wvfs.exists(rellfile) and
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
489 repo.dirstate.normalize(lfile) not in repo[None]):
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
490 wvfs.unlinkpath(rellfile)
20062
452f68738f69 largefiles: inline _updatelfile, prepare for further refactorings
Mads Kiilerich <madski@unity3d.com>
parents: 20061
diff changeset
491 removed += 1
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
492
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
493 # largefile processing might be slow and be interrupted - be prepared
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
494 lfdirstate.write()
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
495
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
496 if lfiles:
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
497 statuswriter(_('getting changed largefiles\n'))
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
498 cachelfiles(ui, repo, None, lfiles)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
499
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
500 for lfile in lfiles:
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
501 update1 = 0
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
502
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
503 expecthash = update.get(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
504 if expecthash:
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
505 if not lfutil.copyfromcache(repo, expecthash, lfile):
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
506 # failed ... but already removed and set to normallookup
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
507 continue
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
508 # Synchronize largefile dirstate to the last modified
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
509 # time of the file
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
510 lfdirstate.normal(lfile)
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
511 update1 = 1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
512
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
513 # copy the state of largefile standin from the repository's
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
514 # dirstate to its state in the lfdirstate.
28559
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
515 rellfile = lfile
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
516 relstandin = lfutil.standin(lfile)
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
517 if wvfs.exists(relstandin):
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
518 mode = wvfs.stat(relstandin).st_mode
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
519 if mode != wvfs.stat(rellfile).st_mode:
30863ca01c6b largefiles: replace invocation of os.path module by vfs in lfcommands.py
liscju <piotr.listkiewicz@gmail.com>
parents: 28464
diff changeset
520 wvfs.chmod(rellfile, mode)
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
521 update1 = 1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
522
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
523 updated += update1
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 20062
diff changeset
524
22095
cb62d77c7a01 largefiles: factor out synchronization of lfdirstate for future use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 21934
diff changeset
525 lfutil.synclfdirstate(repo, lfdirstate, lfile, normallookup)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
526
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
527 lfdirstate.write()
23189
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
528 if lfiles:
fb139f5553d6 largefiles: get function to write status messages via "getstatuswriter()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23139
diff changeset
529 statuswriter(_('%d largefiles updated, %d removed\n') % (updated,
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
530 removed))
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
531
21242
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
532 @command('lfpull',
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
533 [('r', 'rev', [], _('pull largefiles for these revisions'))
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
534 ] + commands.remoteopts,
4c94229c51fb largefiles: declare commands using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21088
diff changeset
535 _('-r REV... [-e CMD] [--remotecmd CMD] [SOURCE]'))
18976
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
536 def lfpull(ui, repo, source="default", **opts):
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
537 """pull largefiles for the specified revisions from the specified source
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
538
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
539 Pull largefiles that are referenced from local changesets but missing
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
540 locally, pulling from a remote repository to the local cache.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
541
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
542 If SOURCE is omitted, the 'default' path will be used.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
543 See :hg:`help urls` for more information.
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
544
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
545 .. container:: verbose
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
546
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
547 Some examples:
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
548
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
549 - pull largefiles for all branch heads::
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
550
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
551 hg lfpull -r "head() and not closed()"
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
552
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
553 - pull largefiles on the default branch::
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
554
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
555 hg lfpull -r "branch(default)"
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
556 """
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
557 repo.lfpullsource = source
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
558
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
559 revs = opts.get('rev', [])
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
560 if not revs:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
561 raise error.Abort(_('no revisions specified'))
18976
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
562 revs = scmutil.revrange(repo, revs)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
563
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
564 numcached = 0
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
565 for rev in revs:
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
566 ui.note(_('pulling largefiles for revision %s\n') % rev)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
567 (cached, missing) = cachelfiles(ui, repo, rev)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
568 numcached += len(cached)
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18974
diff changeset
569 ui.status(_("%d largefiles cached\n") % numcached)