Mercurial > public > mercurial-scm > hg

annotate mercurial/narrowspec.py @ 39531:0d572769046a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
narrowspec: limit patterns to path: and rootfilesin: (BC) Some matcher patterns are computationally expensive and may even have security issues (e.g. evaluating some file sets). For these reasons, we want to limit the types of matcher patterns that can be used in narrow specs and by command line arguments used for defining narrow specs. This commit teaches ``narrowspec.parsepatterns()`` to validate the pattern types against "safe" patterns. Surprisingly, no existing tests broke. So tests for the feature have been added. We also added a function to validate a patterns data structure. This will be used in future commits. Differential Revision: https://phab.mercurial-scm.org/D4522
author Gregory Szorc <gregory.szorc@gmail.com>
date Tue, 11 Sep 2018 15:25:35 -0700
parents 2675d561f5cb
children 8d8e61df8259
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
1 # narrowspec.py - methods for working with a narrow view of a repository
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # Copyright 2017 Google, Inc.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
7
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
8 from __future__ import absolute_import
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
9
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
10 import errno
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
11
36160
9fd8c2a3db5a narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36159
diff changeset
12 from .i18n import _
9fd8c2a3db5a narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36159
diff changeset
13 from . import (
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
14 error,
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
15 match as matchmod,
38869
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
16 repository,
38839
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
17 sparse,
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
18 util,
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
19 )
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
20
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
21 FILENAME = 'narrowspec'
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
22
39531
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
23 # Pattern prefixes that are allowed in narrow patterns. This list MUST
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
24 # only contain patterns that are fast and safe to evaluate. Keep in mind
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
25 # that patterns are supplied by clients and executed on remote servers
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
26 # as part of wire protocol commands.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
27 VALID_PREFIXES = (
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
28 b'path:',
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
29 b'rootfilesin:',
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
30 )
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
31
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
32 def parseserverpatterns(text):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
33 """Parses the narrowspec format that's returned by the server."""
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
34 includepats = set()
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
35 excludepats = set()
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
36
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
37 # We get one entry per line, in the format "<key> <value>".
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
38 # It's OK for value to contain other spaces.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
39 for kp in (l.split(' ', 1) for l in text.splitlines()):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
40 if len(kp) != 2:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
41 raise error.Abort(_('Invalid narrowspec pattern line: "%s"') % kp)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
42 key = kp[0]
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
43 pat = kp[1]
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
44 if key == 'include':
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
45 includepats.add(pat)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
46 elif key == 'exclude':
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
47 excludepats.add(pat)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
48 else:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
49 raise error.Abort(_('Invalid key "%s" in server response') % key)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
50
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
51 return includepats, excludepats
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
52
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
53 def normalizesplitpattern(kind, pat):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
54 """Returns the normalized version of a pattern and kind.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
55
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
56 Returns a tuple with the normalized kind and normalized pattern.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
57 """
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
58 pat = pat.rstrip('/')
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
59 _validatepattern(pat)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
60 return kind, pat
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
61
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
62 def _numlines(s):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
63 """Returns the number of lines in s, including ending empty lines."""
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
64 # We use splitlines because it is Unicode-friendly and thus Python 3
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
65 # compatible. However, it does not count empty lines at the end, so trick
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
66 # it by adding a character at the end.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
67 return len((s + 'x').splitlines())
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
68
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
69 def _validatepattern(pat):
36098
9c55bbc29dcf narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents: 36079
diff changeset
70 """Validates the pattern and aborts if it is invalid.
9c55bbc29dcf narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents: 36079
diff changeset
71
9c55bbc29dcf narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents: 36079
diff changeset
72 Patterns are stored in the narrowspec as newline-separated
9c55bbc29dcf narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents: 36079
diff changeset
73 POSIX-style bytestring paths. There's no escaping.
9c55bbc29dcf narrowspec: document constraints when validating patterns
Augie Fackler <augie@google.com>
parents: 36079
diff changeset
74 """
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
75
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
76 # We use newlines as separators in the narrowspec file, so don't allow them
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
77 # in patterns.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
78 if _numlines(pat) > 1:
36160
9fd8c2a3db5a narrowspec: move module into core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36159
diff changeset
79 raise error.Abort(_('newlines are not allowed in narrowspec paths'))
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
80
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
81 components = pat.split('/')
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
82 if '.' in components or '..' in components:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
83 raise error.Abort(_('"." and ".." are not allowed in narrowspec paths'))
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
84
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
85 def normalizepattern(pattern, defaultkind='path'):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
86 """Returns the normalized version of a text-format pattern.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
87
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
88 If the pattern has no kind, the default will be added.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
89 """
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
90 kind, pat = matchmod._patsplit(pattern, defaultkind)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
91 return '%s:%s' % normalizesplitpattern(kind, pat)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
92
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
93 def parsepatterns(pats):
39531
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
94 """Parses an iterable of patterns into a typed pattern set.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
95
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
96 Patterns are assumed to be ``path:`` if no prefix is present.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
97 For safety and performance reasons, only some prefixes are allowed.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
98 See ``validatepatterns()``.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
99
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
100 This function should be used on patterns that come from the user to
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
101 normalize and validate them to the internal data structure used for
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
102 representing patterns.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
103 """
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
104 res = {normalizepattern(orig) for orig in pats}
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
105 validatepatterns(res)
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
106 return res
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
107
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
108 def validatepatterns(pats):
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
109 """Validate that patterns are in the expected data structure and format.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
110
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
111 And that is a set of normalized patterns beginning with ``path:`` or
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
112 ``rootfilesin:``.
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
113
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
114 This function should be used to validate internal data structures
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
115 and patterns that are loaded from sources that use the internal,
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
116 prefixed pattern representation (but can't necessarily be fully trusted).
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
117 """
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
118 if not isinstance(pats, set):
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
119 raise error.ProgrammingError('narrow patterns should be a set; '
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
120 'got %r' % pats)
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
121
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
122 for pat in pats:
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
123 if not pat.startswith(VALID_PREFIXES):
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
124 # Use a Mercurial exception because this can happen due to user
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
125 # bugs (e.g. manually updating spec file).
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
126 raise error.Abort(_('invalid prefix on narrow pattern: %s') % pat,
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
127 hint=_('narrow patterns must begin with one of '
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
128 'the following: %s') %
0d572769046a narrowspec: limit patterns to path: and rootfilesin: (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39525
diff changeset
129 ', '.join(VALID_PREFIXES))
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
130
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
131 def format(includes, excludes):
38839
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
132 output = '[include]\n'
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
133 for i in sorted(includes - excludes):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
134 output += i + '\n'
38839
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
135 output += '[exclude]\n'
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
136 for e in sorted(excludes):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
137 output += e + '\n'
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
138 return output
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
139
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
140 def match(root, include=None, exclude=None):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
141 if not include:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
142 # Passing empty include and empty exclude to matchmod.match()
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
143 # gives a matcher that matches everything, so explicitly use
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
144 # the nevermatcher.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
145 return matchmod.never(root, '')
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
146 return matchmod.match(root, '', [], include=include or [],
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
147 exclude=exclude or [])
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
148
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
149 def needsexpansion(includes):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
150 return [i for i in includes if i.startswith('include:')]
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
151
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
152 def load(repo):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
153 try:
38872
576eef1ab43d narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38871
diff changeset
154 spec = repo.svfs.read(FILENAME)
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
155 except IOError as e:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
156 # Treat "narrowspec does not exist" the same as "narrowspec file exists
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
157 # and is empty".
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
158 if e.errno == errno.ENOENT:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
159 return set(), set()
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
160 raise
38839
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
161 # maybe we should care about the profiles returned too
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
162 includepats, excludepats, profiles = sparse.parseconfig(repo.ui, spec,
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
163 'narrow')
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
164 if profiles:
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
165 raise error.Abort(_("including other spec files using '%include' is not"
39525
2675d561f5cb narrowspec: fix a typoed 'supported'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38872
diff changeset
166 " supported in narrowspec"))
38839
f64ebe7d2259 narrowspec: use sparse.parseconfig() to parse narrowspec file (BC)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38836
diff changeset
167 return includepats, excludepats
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
168
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
169 def save(repo, includepats, excludepats):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
170 spec = format(includepats, excludepats)
38872
576eef1ab43d narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38871
diff changeset
171 repo.svfs.write(FILENAME, spec)
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
172
38869
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
173 def savebackup(repo, backupname):
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
174 if repository.NARROW_REQUIREMENT not in repo.requirements:
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
175 return
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
176 vfs = repo.vfs
38836
fed6fe856333 narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36470
diff changeset
177 vfs.tryunlink(backupname)
38872
576eef1ab43d narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38871
diff changeset
178 util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname), hardlink=True)
38836
fed6fe856333 narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36470
diff changeset
179
38869
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
180 def restorebackup(repo, backupname):
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
181 if repository.NARROW_REQUIREMENT not in repo.requirements:
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
182 return
38872
576eef1ab43d narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Martin von Zweigbergk <martinvonz@google.com>
parents: 38871
diff changeset
183 util.rename(repo.vfs.join(backupname), repo.svfs.join(FILENAME))
38836
fed6fe856333 narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36470
diff changeset
184
38869
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
185 def clearbackup(repo, backupname):
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
186 if repository.NARROW_REQUIREMENT not in repo.requirements:
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
187 return
ad24b581e4d9 narrow: call narrowspec.{save,restore,clear}backup directly
Martin von Zweigbergk <martinvonz@google.com>
parents: 38840
diff changeset
188 repo.vfs.unlink(backupname)
38836
fed6fe856333 narrow: extract part of narrowspec backup to core
Martin von Zweigbergk <martinvonz@google.com>
parents: 36470
diff changeset
189
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
190 def restrictpatterns(req_includes, req_excludes, repo_includes, repo_excludes):
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
191 r""" Restricts the patterns according to repo settings,
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
192 results in a logical AND operation
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
193
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
194 :param req_includes: requested includes
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
195 :param req_excludes: requested excludes
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
196 :param repo_includes: repo includes
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
197 :param repo_excludes: repo excludes
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
198 :return: include patterns, exclude patterns, and invalid include patterns.
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
199
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
200 >>> restrictpatterns({'f1','f2'}, {}, ['f1'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
201 (set(['f1']), {}, [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
202 >>> restrictpatterns({'f1'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
203 (set(['f1']), {}, [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
204 >>> restrictpatterns({'f1/fc1', 'f3/fc3'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
205 (set(['f1/fc1']), {}, [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
206 >>> restrictpatterns({'f1_fc1'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
207 ([], set(['path:.']), [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
208 >>> restrictpatterns({'f1/../f2/fc2'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
209 (set(['f2/fc2']), {}, [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
210 >>> restrictpatterns({'f1/../f3/fc3'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
211 ([], set(['path:.']), [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
212 >>> restrictpatterns({'f1/$non_exitent_var'}, {}, ['f1','f2'], [])
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
213 (set(['f1/$non_exitent_var']), {}, [])
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
214 """
36099
b8bbe589fd47 narrowspec: consistently use set() to copy sets
Augie Fackler <augie@google.com>
parents: 36098
diff changeset
215 res_excludes = set(req_excludes)
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
216 res_excludes.update(repo_excludes)
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
217 invalid_includes = []
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
218 if not req_includes:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
219 res_includes = set(repo_includes)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
220 elif 'path:.' not in repo_includes:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
221 res_includes = []
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
222 for req_include in req_includes:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
223 req_include = util.expandpath(util.normpath(req_include))
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
224 if req_include in repo_includes:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
225 res_includes.append(req_include)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
226 continue
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
227 valid = False
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
228 for repo_include in repo_includes:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
229 if req_include.startswith(repo_include + '/'):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
230 valid = True
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
231 res_includes.append(req_include)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
232 break
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
233 if not valid:
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
234 invalid_includes.append(req_include)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
235 if len(res_includes) == 0:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
236 res_excludes = {'path:.'}
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
237 else:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
238 res_includes = set(res_includes)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
239 else:
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
240 res_includes = set(req_includes)
36100
8fd0a9e2d7e9 narrow: make restrictpatterns a little more idiomatic
Augie Fackler <augie@google.com>
parents: 36099
diff changeset
241 return res_includes, res_excludes, invalid_includes