Mercurial > public > mercurial-scm > hg

annotate hgext/lfs/wrapper.py @ 43089:c59eb1560c44

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
py3: manually import getattr where it is needed The march continues. Differential Revision: https://phab.mercurial-scm.org/D7009
author Gregory Szorc <gregory.szorc@gmail.com>
date Sun, 06 Oct 2019 16:55:18 -0400
parents 66f2cc210a29
children 649d3ac37a12
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
1 # wrapper.py - methods wrapping core mercurial logic
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
2 #
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
3 # Copyright 2017 Facebook, Inc.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
4 #
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
7
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
8 from __future__ import absolute_import
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
9
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
10 import hashlib
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
11
35099
b8e5fb8d2389 lfs: quiesce check-module-import warnings
Matt Harbison <matt_harbison@yahoo.com>
parents: 35098
diff changeset
12 from mercurial.i18n import _
36604
f6b9f9ddd0de lfs: convert hexdigest to bytes using sysbytes
Augie Fackler <augie@google.com>
parents: 36603
diff changeset
13 from mercurial.node import bin, hex, nullid, short
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
14 from mercurial.pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
15 getattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
16 setattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
17 )
35099
b8e5fb8d2389 lfs: quiesce check-module-import warnings
Matt Harbison <matt_harbison@yahoo.com>
parents: 35098
diff changeset
18
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
19 from mercurial import (
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
20 bundle2,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
21 changegroup,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
22 cmdutil,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
23 context,
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
24 error,
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
25 exchange,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
26 exthelper,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
27 localrepo,
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
28 revlog,
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
29 scmutil,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
30 upgrade,
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
31 util,
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
32 vfs as vfsmod,
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
33 wireprotov1server,
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
34 )
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
35
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
36 from mercurial.interfaces import repository
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42775
diff changeset
37
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36604
diff changeset
38 from mercurial.utils import (
39878
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39856
diff changeset
39 storageutil,
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36604
diff changeset
40 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36604
diff changeset
41 )
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36604
diff changeset
42
35363
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
43 from ..largefiles import lfutil
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
44
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
45 from . import (
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
46 blobstore,
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
47 pointer,
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
48 )
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
49
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
50 eh = exthelper.exthelper()
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
51
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
52
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
53 @eh.wrapfunction(localrepo, b'makefilestorage')
39851
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
54 def localrepomakefilestorage(orig, requirements, features, **kwargs):
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
55 if b'lfs' in requirements:
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
56 features.add(repository.REPO_FEATURE_LFS)
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
57
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
58 return orig(requirements=requirements, features=features, **kwargs)
1f7b3b980af8 lfs: add repository feature denoting the use of LFS
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39849
diff changeset
59
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
60
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
61 @eh.wrapfunction(changegroup, b'allsupportedversions')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
62 def allsupportedversions(orig, ui):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
63 versions = orig(ui)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
64 versions.add(b'03')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
65 return versions
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
66
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
67
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
68 @eh.wrapfunction(wireprotov1server, b'_capabilities')
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
69 def _capabilities(orig, repo, proto):
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
70 '''Wrap server command to announce lfs server capability'''
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
71 caps = orig(repo, proto)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
72 if util.safehasattr(repo.svfs, b'lfslocalblobstore'):
40324
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
73 # Advertise a slightly different capability when lfs is *required*, so
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
74 # that the client knows it MUST load the extension. If lfs is not
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
75 # required on the server, there's no reason to autoload the extension
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
76 # on the client.
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
77 if b'lfs' in repo.requirements:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
78 caps.append(b'lfs-serve')
40324
6637b079ae45 lfs: autoload the extension when cloning from repo with lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 39949
diff changeset
79
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
80 caps.append(b'lfs')
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
81 return caps
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
82
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
83
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
84 def bypasscheckhash(self, text):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
85 return False
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
86
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
87
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
88 def readfromstore(self, text):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
89 """Read filelog content from local blobstore transform for flagprocessor.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
90
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
91 Default tranform for flagprocessor, returning contents from blobstore.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
92 Returns a 2-typle (text, validatehash) where validatehash is True as the
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
93 contents of the blobstore should be checked using checkhash.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
94 """
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
95 p = pointer.deserialize(text)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
96 oid = p.oid()
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
97 store = self.opener.lfslocalblobstore
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
98 if not store.has(oid):
35568
ebf14075a5c1 lfs: improve the error message for a missing remote blob
Matt Harbison <matt_harbison@yahoo.com>
parents: 35553
diff changeset
99 p.filename = self.filename
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
100 self.opener.lfsremoteblobstore.readbatch([p], store)
35476
417e8e040102 lfs: verify lfs object content when transferring to and from the remote store
Matt Harbison <matt_harbison@yahoo.com>
parents: 35473
diff changeset
101
417e8e040102 lfs: verify lfs object content when transferring to and from the remote store
Matt Harbison <matt_harbison@yahoo.com>
parents: 35473
diff changeset
102 # The caller will validate the content
417e8e040102 lfs: verify lfs object content when transferring to and from the remote store
Matt Harbison <matt_harbison@yahoo.com>
parents: 35473
diff changeset
103 text = store.read(oid, verify=False)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
104
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
105 # pack hg filelog metadata
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
106 hgmeta = {}
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
107 for k in p.keys():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
108 if k.startswith(b'x-hg-'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
109 name = k[len(b'x-hg-') :]
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
110 hgmeta[name] = p[k]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
111 if hgmeta or text.startswith(b'\1\n'):
39878
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39856
diff changeset
112 text = storageutil.packmeta(hgmeta, text)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
113
42985
bd5858c28bbe flagprocessors: have the read transform function return side data (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
114 return (text, True, {})
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
115
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
116
42988
f4caf910669e flagprocessors: writetransform function take side data as parameter (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42985
diff changeset
117 def writetostore(self, text, sidedata):
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
118 # hg filelog metadata (includes rename, etc)
39878
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39856
diff changeset
119 hgmeta, offset = storageutil.parsemeta(text)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
120 if offset and offset > 0:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
121 # lfs blob does not contain hg filelog metadata
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
122 text = text[offset:]
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
123
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
124 # git-lfs only supports sha256
36604
f6b9f9ddd0de lfs: convert hexdigest to bytes using sysbytes
Augie Fackler <augie@google.com>
parents: 36603
diff changeset
125 oid = hex(hashlib.sha256(text).digest())
35553
a77418095530 lfs: remove the verification option when writing to the local store
Matt Harbison <matt_harbison@yahoo.com>
parents: 35506
diff changeset
126 self.opener.lfslocalblobstore.write(oid, text)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
127
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
128 # replace contents with metadata
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
129 longoid = b'sha256:%s' % oid
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
130 metadata = pointer.gitlfspointer(oid=longoid, size=b'%d' % len(text))
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
131
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
132 # by default, we expect the content to be binary. however, LFS could also
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
133 # be used for non-binary content. add a special entry for non-binary data.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
134 # this will be used by filectx.isbinary().
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36604
diff changeset
135 if not stringutil.binary(text):
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
136 # not hg filelog metadata (affecting commit hash), no "x-hg-" prefix
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
137 metadata[b'x-is-binary'] = b'0'
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
138
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
139 # translate hg filelog metadata to lfs metadata with "x-hg-" prefix
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
140 if hgmeta is not None:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
141 for k, v in hgmeta.iteritems():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
142 metadata[b'x-hg-%s' % k] = v
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
143
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
144 rawtext = metadata.serialize()
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
145 return (rawtext, False)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
146
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
147
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
148 def _islfs(rlog, node=None, rev=None):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
149 if rev is None:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
150 if node is None:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
151 # both None - likely working copy content where node is not ready
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
152 return False
39855
62a532045e71 lfs: access revlog directly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39853
diff changeset
153 rev = rlog._revlog.rev(node)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
154 else:
39855
62a532045e71 lfs: access revlog directly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39853
diff changeset
155 node = rlog._revlog.node(rev)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
156 if node == nullid:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
157 return False
39855
62a532045e71 lfs: access revlog directly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39853
diff changeset
158 flags = rlog._revlog.flags(rev)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
159 return bool(flags & revlog.REVIDX_EXTSTORED)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
160
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
161
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
162 # Wrapping may also be applied by remotefilelog
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
163 def filelogaddrevision(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
164 orig,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
165 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
166 text,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
167 transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
168 link,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
169 p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
170 p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
171 cachedelta=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
172 node=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
173 flags=revlog.REVIDX_DEFAULT_FLAGS,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
174 **kwds
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
175 ):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
176 # The matcher isn't available if reposetup() wasn't called.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
177 lfstrack = self._revlog.opener.options.get(b'lfstrack')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
178
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
179 if lfstrack:
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
180 textlen = len(text)
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
181 # exclude hg rename meta from file size
39878
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39856
diff changeset
182 meta, offset = storageutil.parsemeta(text)
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
183 if offset:
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
184 textlen -= offset
35648
588d02d9208a lfs: always exclude '.hg*' text files
Matt Harbison <matt_harbison@yahoo.com>
parents: 35624
diff changeset
185
39856
96838b620b9c filelog: store filename directly on revlog instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39855
diff changeset
186 if lfstrack(self._revlog.filename, textlen):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
187 flags |= revlog.REVIDX_EXTSTORED
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
188
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
189 return orig(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
190 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
191 text,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
192 transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
193 link,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
194 p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
195 p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
196 cachedelta=cachedelta,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
197 node=node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
198 flags=flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
199 **kwds
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
200 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
201
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
202
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
203 # Wrapping may also be applied by remotefilelog
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
204 def filelogrenamed(orig, self, node):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
205 if _islfs(self, node):
42775
f972f43badb3 rawdata: update callers in lfs' wrapper
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41279
diff changeset
206 rawtext = self._revlog.rawdata(node)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
207 if not rawtext:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
208 return False
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
209 metadata = pointer.deserialize(rawtext)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
210 if b'x-hg-copy' in metadata and b'x-hg-copyrev' in metadata:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
211 return metadata[b'x-hg-copy'], bin(metadata[b'x-hg-copyrev'])
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
212 else:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
213 return False
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
214 return orig(self, node)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
215
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
216
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
217 # Wrapping may also be applied by remotefilelog
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
218 def filelogsize(orig, self, rev):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
219 if _islfs(self, rev=rev):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
220 # fast path: use lfs metadata to answer size
42775
f972f43badb3 rawdata: update callers in lfs' wrapper
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41279
diff changeset
221 rawtext = self._revlog.rawdata(rev)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
222 metadata = pointer.deserialize(rawtext)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
223 return int(metadata[b'size'])
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
224 return orig(self, rev)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
225
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
226
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
227 @eh.wrapfunction(context.basefilectx, b'cmp')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
228 def filectxcmp(orig, self, fctx):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
229 """returns True if text is different than fctx"""
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
230 # some fctx (ex. hg-git) is not based on basefilectx and do not have islfs
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
231 if self.islfs() and getattr(fctx, 'islfs', lambda: False)():
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
232 # fast path: check LFS oid
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
233 p1 = pointer.deserialize(self.rawdata())
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
234 p2 = pointer.deserialize(fctx.rawdata())
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
235 return p1.oid() != p2.oid()
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
236 return orig(self, fctx)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
237
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
238
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
239 @eh.wrapfunction(context.basefilectx, b'isbinary')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
240 def filectxisbinary(orig, self):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
241 if self.islfs():
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
242 # fast path: use lfs metadata to answer isbinary
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
243 metadata = pointer.deserialize(self.rawdata())
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
244 # if lfs metadata says nothing, assume it's binary by default
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
245 return bool(int(metadata.get(b'x-is-binary', 1)))
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
246 return orig(self)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
247
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
248
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
249 def filectxislfs(self):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
250 return _islfs(self.filelog(), self.filenode())
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
251
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
252
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
253 @eh.wrapfunction(cmdutil, b'_updatecatformatter')
35663
a985834961f7 lfs: allow the pointer file to be viewed with `hg cat -T '{rawdata}'`
Matt Harbison <matt_harbison@yahoo.com>
parents: 35648
diff changeset
254 def _updatecatformatter(orig, fm, ctx, matcher, path, decode):
a985834961f7 lfs: allow the pointer file to be viewed with `hg cat -T '{rawdata}'`
Matt Harbison <matt_harbison@yahoo.com>
parents: 35648
diff changeset
255 orig(fm, ctx, matcher, path, decode)
a985834961f7 lfs: allow the pointer file to be viewed with `hg cat -T '{rawdata}'`
Matt Harbison <matt_harbison@yahoo.com>
parents: 35648
diff changeset
256 fm.data(rawdata=ctx[path].rawdata())
a985834961f7 lfs: allow the pointer file to be viewed with `hg cat -T '{rawdata}'`
Matt Harbison <matt_harbison@yahoo.com>
parents: 35648
diff changeset
257
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
258
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
259 @eh.wrapfunction(scmutil, b'wrapconvertsink')
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
260 def convertsink(orig, sink):
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
261 sink = orig(sink)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
262 if sink.repotype == b'hg':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
263
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
264 class lfssink(sink.__class__):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
265 def putcommit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
266 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
267 files,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
268 copies,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
269 parents,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
270 commit,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
271 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
272 revmap,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
273 full,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
274 cleanp2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
275 ):
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
276 pc = super(lfssink, self).putcommit
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
277 node = pc(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
278 files,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
279 copies,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
280 parents,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
281 commit,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
282 source,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
283 revmap,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
284 full,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
285 cleanp2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
286 )
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
287
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
288 if b'lfs' not in self.repo.requirements:
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
289 ctx = self.repo[node]
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
290
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
291 # The file list may contain removed files, so check for
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
292 # membership before assuming it is in the context.
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
293 if any(f in ctx and ctx[f].islfs() for f, n in files):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
294 self.repo.requirements.add(b'lfs')
35178
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
295 self.repo._writerequirements()
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
296
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
297 return node
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
298
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
299 sink.__class__ = lfssink
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
300
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
301 return sink
f8f939a2926c lfs: add a repo requirement for this extension when converting to lfs
Matt Harbison <matt_harbison@yahoo.com>
parents: 35099
diff changeset
302
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
303
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
304 # bundlerepo uses "vfsmod.readonlyvfs(othervfs)", we need to make sure lfs
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
305 # options and blob stores are passed from othervfs to the new readonlyvfs.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
306 @eh.wrapfunction(vfsmod.readonlyvfs, b'__init__')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
307 def vfsinit(orig, self, othervfs):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
308 orig(self, othervfs)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
309 # copy lfs related options
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
310 for k, v in othervfs.options.items():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
311 if k.startswith(b'lfs'):
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
312 self.options[k] = v
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
313 # also copy lfs blobstores. note: this can run before reposetup, so lfs
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
314 # blobstore attributes are not always ready at this time.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
315 for name in [b'lfslocalblobstore', b'lfsremoteblobstore']:
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
316 if util.safehasattr(othervfs, name):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
317 setattr(self, name, getattr(othervfs, name))
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
318
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
319
37762
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
320 def _prefetchfiles(repo, revs, match):
35922
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
321 """Ensure that required LFS blobs are present, fetching them as a group if
36142
60dd840a7fdb merge: invoke scmutil.fileprefetchhooks() prior to applying updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 36138
diff changeset
322 needed."""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
323 if not util.safehasattr(repo.svfs, b'lfslocalblobstore'):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
324 return
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
325
35922
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
326 pointers = []
37762
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
327 oids = set()
35922
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
328 localstore = repo.svfs.lfslocalblobstore
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
329
37762
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
330 for rev in revs:
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
331 ctx = repo[rev]
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
332 for f in ctx.walk(match):
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
333 p = pointerfromctx(ctx, f)
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
334 if p and p.oid() not in oids and not localstore.has(p.oid()):
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
335 p.filename = f
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
336 pointers.append(p)
7269b87f817c scmutil: teach the file prefetch hook to handle multiple commits
Matt Harbison <matt_harbison@yahoo.com>
parents: 37564
diff changeset
337 oids.add(p.oid())
35922
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
338
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
339 if pointers:
37518
092eff6833a7 lfs: infer the blob store URL from paths.default
Matt Harbison <matt_harbison@yahoo.com>
parents: 37442
diff changeset
340 # Recalculating the repo store here allows 'paths.default' that is set
092eff6833a7 lfs: infer the blob store URL from paths.default
Matt Harbison <matt_harbison@yahoo.com>
parents: 37442
diff changeset
341 # on the repo by a clone command to be used for the update.
092eff6833a7 lfs: infer the blob store URL from paths.default
Matt Harbison <matt_harbison@yahoo.com>
parents: 37442
diff changeset
342 blobstore.remote(repo).readbatch(pointers, localstore)
35922
0b79f99fd7b0 lfs: prefetch lfs blobs when applying merge updates
Matt Harbison <matt_harbison@yahoo.com>
parents: 35921
diff changeset
343
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
344
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
345 def _canskipupload(repo):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
346 # Skip if this hasn't been passed to reposetup()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
347 if not util.safehasattr(repo.svfs, b'lfsremoteblobstore'):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
348 return True
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
349
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
350 # if remotestore is a null store, upload is a no-op and can be skipped
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
351 return isinstance(repo.svfs.lfsremoteblobstore, blobstore._nullremote)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
352
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
353
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
354 def candownload(repo):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
355 # Skip if this hasn't been passed to reposetup()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
356 if not util.safehasattr(repo.svfs, b'lfsremoteblobstore'):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
357 return False
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
358
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
359 # if remotestore is a null store, downloads will lead to nothing
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
360 return not isinstance(repo.svfs.lfsremoteblobstore, blobstore._nullremote)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
361
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
362
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
363 def uploadblobsfromrevs(repo, revs):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
364 '''upload lfs blobs introduced by revs
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
365
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
366 Note: also used by other extensions e. g. infinitepush. avoid renaming.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
367 '''
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
368 if _canskipupload(repo):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
369 return
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
370 pointers = extractpointers(repo, revs)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
371 uploadblobs(repo, pointers)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
372
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
373
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
374 def prepush(pushop):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
375 """Prepush hook.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
376
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
377 Read through the revisions to push, looking for filelog entries that can be
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
378 deserialized into metadata so that we can block the push on their upload to
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
379 the remote blobstore.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
380 """
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
381 return uploadblobsfromrevs(pushop.repo, pushop.outgoing.missing)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
382
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
383
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
384 @eh.wrapfunction(exchange, b'push')
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
385 def push(orig, repo, remote, *args, **kwargs):
37564
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
386 """bail on push if the extension isn't enabled on remote when needed, and
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
387 update the remote store based on the destination path."""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
388 if b'lfs' in repo.requirements:
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
389 # If the remote peer is for a local repo, the requirement tests in the
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
390 # base class method enforce lfs support. Otherwise, some revisions in
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
391 # this repo use lfs, and the remote repo needs the extension loaded.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
392 if not remote.local() and not remote.capable(b'lfs'):
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
393 # This is a copy of the message in exchange.push() when requirements
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
394 # are missing between local repos.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
395 m = _(b"required features are not supported in the destination: %s")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
396 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
397 m % b'lfs', hint=_(b'enable the lfs extension on the server')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
398 )
37564
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
399
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
400 # Repositories where this extension is disabled won't have the field.
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
401 # But if there's a requirement, then the extension must be loaded AND
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
402 # there may be blobs to push.
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
403 remotestore = repo.svfs.lfsremoteblobstore
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
404 try:
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
405 repo.svfs.lfsremoteblobstore = blobstore.remote(repo, remote.url())
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
406 return orig(repo, remote, *args, **kwargs)
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
407 finally:
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
408 repo.svfs.lfsremoteblobstore = remotestore
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
409 else:
31a4ea773369 lfs: infer the blob store URL from an explicit push dest or default-push
Matt Harbison <matt_harbison@yahoo.com>
parents: 37518
diff changeset
410 return orig(repo, remote, *args, **kwargs)
35506
fa865878a849 lfs: show a friendly message when pushing lfs to a server without lfs enabled
Matt Harbison <matt_harbison@yahoo.com>
parents: 35505
diff changeset
411
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
412
41048
84d61fdcefa5 lfs: convert to using exthelper to wrap functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 40386
diff changeset
413 # when writing a bundle via "hg bundle" command, upload related LFS blobs
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
414 @eh.wrapfunction(bundle2, b'writenewbundle')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
415 def writenewbundle(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
416 orig, ui, repo, source, filename, bundletype, outgoing, *args, **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
417 ):
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
418 """upload LFS blobs added by outgoing revisions on 'hg bundle'"""
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
419 uploadblobsfromrevs(repo, outgoing.missing)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
420 return orig(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
421 ui, repo, source, filename, bundletype, outgoing, *args, **kwargs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
422 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
423
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
424
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
425 def extractpointers(repo, revs):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
426 """return a list of lfs pointers added by given revs"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
427 repo.ui.debug(b'lfs: computing set of blobs to upload\n')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
428 pointers = {}
39270
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
429
39389
b26350d9d7b5 lfs: use a context manager to control the progress bar lifetime
Matt Harbison <matt_harbison@yahoo.com>
parents: 39270
diff changeset
430 makeprogress = repo.ui.makeprogress
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
431 with makeprogress(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
432 _(b'lfs search'), _(b'changesets'), len(revs)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
433 ) as progress:
39270
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
434 for r in revs:
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
435 ctx = repo[r]
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
436 for p in pointersfromctx(ctx).values():
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
437 pointers[p.oid()] = p
37e56607cbb9 lfs: add a progress bar when searching for blobs to upload
Matt Harbison <matt_harbison@yahoo.com>
parents: 38178
diff changeset
438 progress.increment()
39949
e1f97179a3f5 lfs: explicitly name a key when sorting blob pointers
Matt Harbison <matt_harbison@yahoo.com>
parents: 39878
diff changeset
439 return sorted(pointers.values(), key=lambda p: p.oid())
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
440
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
441
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
442 def pointerfromctx(ctx, f, removed=False):
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
443 """return a pointer for the named file from the given changectx, or None if
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
444 the file isn't LFS.
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
445
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
446 Optionally, the pointer for a file deleted from the context can be returned.
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
447 Since no such pointer is actually stored, and to distinguish from a non LFS
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
448 file, this pointer is represented by an empty dict.
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
449 """
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
450 _ctx = ctx
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
451 if f not in ctx:
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
452 if not removed:
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
453 return None
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
454 if f in ctx.p1():
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
455 _ctx = ctx.p1()
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
456 elif f in ctx.p2():
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
457 _ctx = ctx.p2()
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
458 else:
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
459 return None
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
460 fctx = _ctx[f]
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
461 if not _islfs(fctx.filelog(), fctx.filenode()):
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
462 return None
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
463 try:
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
464 p = pointer.deserialize(fctx.rawdata())
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
465 if ctx == _ctx:
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
466 return p
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
467 return {}
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
468 except pointer.InvalidPointer as ex:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
469 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
470 _(b'lfs: corrupted pointer (%s@%s): %s\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
471 % (f, short(_ctx.node()), ex)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
472 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
473
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
474
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
475 def pointersfromctx(ctx, removed=False):
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
476 """return a dict {path: pointer} for given single changectx.
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
477
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
478 If ``removed`` == True and the LFS file was removed from ``ctx``, the value
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
479 stored for the path is an empty dict.
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
480 """
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
481 result = {}
40386
4a81d82474e9 lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
Matt Harbison <matt_harbison@yahoo.com>
parents: 40368
diff changeset
482 m = ctx.repo().narrowmatch()
4a81d82474e9 lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
Matt Harbison <matt_harbison@yahoo.com>
parents: 40368
diff changeset
483
4a81d82474e9 lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
Matt Harbison <matt_harbison@yahoo.com>
parents: 40368
diff changeset
484 # TODO: consider manifest.fastread() instead
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
485 for f in ctx.files():
40386
4a81d82474e9 lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
Matt Harbison <matt_harbison@yahoo.com>
parents: 40368
diff changeset
486 if not m(f):
4a81d82474e9 lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
Matt Harbison <matt_harbison@yahoo.com>
parents: 40368
diff changeset
487 continue
35998
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
488 p = pointerfromctx(ctx, f, removed=removed)
dce43aaaf209 lfs: allow a pointer to be extracted from a context that removes the file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35924
diff changeset
489 if p is not None:
35921
47e737d27e01 lfs: factor out a method for extracting the pointer of a single file
Matt Harbison <matt_harbison@yahoo.com>
parents: 35799
diff changeset
490 result[f] = p
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
491 return result
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
492
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
493
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
494 def uploadblobs(repo, pointers):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
495 """upload given pointers from local blobstore"""
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
496 if not pointers:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
497 return
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
498
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
499 remoteblob = repo.svfs.lfsremoteblobstore
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
500 remoteblob.writebatch(pointers, repo.svfs.lfslocalblobstore)
35346
9eb19b13e92a lfs: allow to run 'debugupgraderepo' on repo with largefiles
Boris Feld <boris.feld@octobus.net>
parents: 35215
diff changeset
501
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
502
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
503 @eh.wrapfunction(upgrade, b'_finishdatamigration')
35363
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
504 def upgradefinishdatamigration(orig, ui, srcrepo, dstrepo, requirements):
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
505 orig(ui, srcrepo, dstrepo, requirements)
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
506
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
507 # Skip if this hasn't been passed to reposetup()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
508 if util.safehasattr(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
509 srcrepo.svfs, b'lfslocalblobstore'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
510 ) and util.safehasattr(dstrepo.svfs, b'lfslocalblobstore'):
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
511 srclfsvfs = srcrepo.svfs.lfslocalblobstore.vfs
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
512 dstlfsvfs = dstrepo.svfs.lfslocalblobstore.vfs
35363
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
513
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
514 for dirpath, dirs, files in srclfsvfs.walk():
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
515 for oid in files:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
516 ui.write(_(b'copying lfs blob %s\n') % oid)
38178
3790efb388ca lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
Matt Harbison <matt_harbison@yahoo.com>
parents: 37762
diff changeset
517 lfutil.link(srclfsvfs.join(oid), dstlfsvfs.join(oid))
35363
b0ba1539af01 lfs: restore the local blob store after a repo upgrade
Matt Harbison <matt_harbison@yahoo.com>
parents: 35346
diff changeset
518
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42988
diff changeset
519
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
520 @eh.wrapfunction(upgrade, b'preservedrequirements')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
521 @eh.wrapfunction(upgrade, b'supporteddestrequirements')
35346
9eb19b13e92a lfs: allow to run 'debugupgraderepo' on repo with largefiles
Boris Feld <boris.feld@octobus.net>
parents: 35215
diff changeset
522 def upgraderequirements(orig, repo):
9eb19b13e92a lfs: allow to run 'debugupgraderepo' on repo with largefiles
Boris Feld <boris.feld@octobus.net>
parents: 35215
diff changeset
523 reqs = orig(repo)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
524 if b'lfs' in repo.requirements:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
525 reqs.add(b'lfs')
35346
9eb19b13e92a lfs: allow to run 'debugupgraderepo' on repo with largefiles
Boris Feld <boris.feld@octobus.net>
parents: 35215
diff changeset
526 return reqs