Mercurial > public > mercurial-scm > hg-stable

annotate hgext/lfs/pointer.py @ 43105:649d3ac37a12

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
py3: define and use pycompat.iteritems() for hgext/ .iteritems() -> .items() is the last source transform being performed. But it is also the most widely used. This commit adds a pycompat.iteritems symbol and imports it in place of .iteritems() for usage in hgext/. I chose to stop at just hgext/ because the patch will be large and it is an easy boundary to stop at since we can disable source transformation on a per-package basis. There are places where the type does implement items() and we could call items() directly. However, this would require critical thought and I thought it would be easier to just blindly change the code. We know which call sites need to be audited in the future because they have "pycompat.iteritems." With this change, we no longer perform source transformation on hgext! Differential Revision: https://phab.mercurial-scm.org/D7014
author Gregory Szorc <gregory.szorc@gmail.com>
date Sun, 06 Oct 2019 19:25:18 -0400
parents 687b865b95ad
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
1 # pointer.py - Git-LFS pointer serialization
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
2 #
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
3 # Copyright 2017 Facebook, Inc.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
4 #
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
7
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
8 from __future__ import absolute_import
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
9
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
10 import re
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
11
35122
b8e5fb8d2389 lfs: quiesce check-module-import warnings
Matt Harbison <matt_harbison@yahoo.com>
parents: 35121
diff changeset
12 from mercurial.i18n import _
b8e5fb8d2389 lfs: quiesce check-module-import warnings
Matt Harbison <matt_harbison@yahoo.com>
parents: 35121
diff changeset
13
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
14 from mercurial import (
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
15 error,
36631
e30be4d2ac60 lfs: use byteskwargs() on some **kwargs for python 3 compat
Augie Fackler <augie@google.com>
parents: 36629
diff changeset
16 pycompat,
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
17 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
18 from mercurial.utils import stringutil
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
19
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
20
39793
b63dee7bd0d9 global: replace most uses of RevlogError with StorageError (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38479
diff changeset
21 class InvalidPointer(error.StorageError):
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
22 pass
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
23
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
24
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
25 class gitlfspointer(dict):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
26 VERSION = b'https://git-lfs.github.com/spec/v1'
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
27
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
28 def __init__(self, *args, **kwargs):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
29 self[b'version'] = self.VERSION
36631
e30be4d2ac60 lfs: use byteskwargs() on some **kwargs for python 3 compat
Augie Fackler <augie@google.com>
parents: 36629
diff changeset
30 super(gitlfspointer, self).__init__(*args)
e30be4d2ac60 lfs: use byteskwargs() on some **kwargs for python 3 compat
Augie Fackler <augie@google.com>
parents: 36629
diff changeset
31 self.update(pycompat.byteskwargs(kwargs))
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
32
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
33 @classmethod
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
34 def deserialize(cls, text):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
35 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
36 return cls(l.split(b' ', 1) for l in text.splitlines()).validate()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
37 except ValueError: # l.split returns 1 item instead of 2
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
38 raise InvalidPointer(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
39 _(b'cannot parse git-lfs text: %s') % stringutil.pprint(text)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
40 )
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
41
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
42 def serialize(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
43 sortkeyfunc = lambda x: (x[0] != b'version', x)
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
44 items = sorted(pycompat.iteritems(self.validate()), key=sortkeyfunc)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
45 return b''.join(b'%s %s\n' % (k, v) for k, v in items)
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
46
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
47 def oid(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
48 return self[b'oid'].split(b':')[-1]
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
49
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
50 def size(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
51 return int(self[b'size'])
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
52
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
53 # regular expressions used by _validate
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
54 # see https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md
36629
95bd9e396774 lfs: add missing b prefixes on some regular expressions
Augie Fackler <augie@google.com>
parents: 35122
diff changeset
55 _keyre = re.compile(br'\A[a-z0-9.-]+\Z')
95bd9e396774 lfs: add missing b prefixes on some regular expressions
Augie Fackler <augie@google.com>
parents: 35122
diff changeset
56 _valuere = re.compile(br'\A[^\n]*\Z')
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
57 _requiredre = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
58 b'size': re.compile(br'\A[0-9]+\Z'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
59 b'oid': re.compile(br'\Asha256:[0-9a-f]{64}\Z'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
60 b'version': re.compile(br'\A%s\Z' % stringutil.reescape(VERSION)),
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
61 }
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
62
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
63 def validate(self):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
64 """raise InvalidPointer on error. return self if there is no error"""
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
65 requiredcount = 0
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
66 for k, v in pycompat.iteritems(self):
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
67 if k in self._requiredre:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
68 if not self._requiredre[k].match(v):
38194
aa10675c5dd6 lfs: clarify pointer validation error messages
Matt Harbison <matt_harbison@yahoo.com>
parents: 37986
diff changeset
69 raise InvalidPointer(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
70 _(b'unexpected lfs pointer value: %s=%s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
71 % (k, stringutil.pprint(v))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
72 )
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
73 requiredcount += 1
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
74 elif not self._keyre.match(k):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
75 raise InvalidPointer(_(b'unexpected lfs pointer key: %s') % k)
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
76 if not self._valuere.match(v):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
77 raise InvalidPointer(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
78 _(b'unexpected lfs pointer value: %s=%s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
79 % (k, stringutil.pprint(v))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
80 )
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
81 if len(self._requiredre) != requiredcount:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
82 miss = sorted(set(self._requiredre.keys()).difference(self.keys()))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
83 raise InvalidPointer(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
84 _(b'missing lfs pointer keys: %s') % b', '.join(miss)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
85 )
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
86 return self
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
87
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39793
diff changeset
88
35121
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
89 deserialize = gitlfspointer.deserialize