Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/parser.py @ 26117:4dc5b51f38fe
revlog: change generaldelta delta parent heuristic
The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text,
use it, otherwise use prev". This was problematic when a repo contained multiple
branches that were very different. If commits to branch A were pushed, and the
last full text was branch B, it would generate a fulltext. Then if branch B was
pushed, it would generate another fulltext. The problem is that the last
fulltext (and delta'ing against `prev` in general) has no correlation with the
contents of the incoming revision, and therefore will always have degenerate
cases.
According to the blame, that algorithm was chosen to minimize the chain length.
Since there is already code that protects against that (the delta-vs-fulltext
code), and since it has been improved since the original generaldelta algorithm
went in (2011), I believe the chain length criteria will still be preserved.
The new algorithm always diffs against p1 (or p2 if it's closer), unless the
resulting delta will fail the delta-vs-fulltext check, in which case we delta
against prev.
Some before and after stats on manifest.d size.
internal large repo
old heuristic - 2.0 GB
new heuristic - 1.2 GB
mozilla-central
old heuristic - 242 MB
new heuristic - 261 MB
The regression in mozilla central is due to the new heuristic choosing p2r as
the delta when it's closer to the tip. Switching the algorithm to always prefer
p1r brings the size back down (242 MB). This is result of the way in which
mozilla does merges and pushes, and the result could easily swing the other
direction in other repos (depending on if they merge X into Y or Y into X), but
will never be as degenerate as before.
I future patch will address the regression by introducing an optional, even more
aggressive delta heuristic which will knock the mozilla manifest size down
dramatically.
author | Durham Goode <durham@fb.com> |
---|---|
date | Sun, 30 Aug 2015 13:58:11 -0700 |
parents | 7448df709b2e |
children | 87c9c562c37a |
rev | line source |
---|---|
11274 | 1 # parser.py - simple top-down operator precedence parser for mercurial |
2 # | |
3 # Copyright 2010 Matt Mackall <mpm@selenic.com> | |
4 # | |
5 # This software may be used and distributed according to the terms of the | |
6 # GNU General Public License version 2 or any later version. | |
7 | |
11449
05af334bac05
parser: fix URL to effbot
Julian Cowley <julian@lava.net>
parents:
11412
diff
changeset
|
8 # see http://effbot.org/zone/simple-top-down-parsing.htm and |
11274 | 9 # http://eli.thegreenplace.net/2010/01/02/top-down-operator-precedence-parsing/ |
10 # for background | |
11 | |
12 # takes a tokenizer and elements | |
25655
b8b73652c1c9
parser: update documentation about tokenizer and elements
Yuya Nishihara <yuya@tcha.org>
parents:
25654
diff
changeset
|
13 # tokenizer is an iterator that returns (type, value, pos) tuples |
25815
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
14 # elements is a mapping of types to binding strength, primary, prefix, infix |
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
15 # and suffix actions |
11274 | 16 # an action is a tree node name, a tree label, and an optional match |
17500 | 17 # __call__(program) parses program into a labeled tree |
11274 | 18 |
25963
7448df709b2e
parser: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25818
diff
changeset
|
19 from __future__ import absolute_import |
7448df709b2e
parser: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25818
diff
changeset
|
20 |
7448df709b2e
parser: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25818
diff
changeset
|
21 from .i18n import _ |
7448df709b2e
parser: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25818
diff
changeset
|
22 from . import error |
11289
4215ce511134
revset: raise ParseError exceptions
Matt Mackall <mpm@selenic.com>
parents:
11278
diff
changeset
|
23 |
11274 | 24 class parser(object): |
25654
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
25 def __init__(self, elements, methods=None): |
11274 | 26 self._elements = elements |
27 self._methods = methods | |
13176
895f54a79c6e
templater: use the parser.py parser to extend the templater syntax
Matt Mackall <mpm@selenic.com>
parents:
11449
diff
changeset
|
28 self.current = None |
11274 | 29 def _advance(self): |
30 'advance the tokenizer' | |
31 t = self.current | |
25171
d647f97f88dd
parsers: use 'next' instead of try/except
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20778
diff
changeset
|
32 self.current = next(self._iter, None) |
11274 | 33 return t |
25804
f0a77cb6316a
parser: extract function that tests if next token may start new term
Yuya Nishihara <yuya@tcha.org>
parents:
25803
diff
changeset
|
34 def _hasnewterm(self): |
f0a77cb6316a
parser: extract function that tests if next token may start new term
Yuya Nishihara <yuya@tcha.org>
parents:
25803
diff
changeset
|
35 'True if next token may start new term' |
25815
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
36 return any(self._elements[self.current[0]][1:3]) |
25802
cc741c76b26a
parser: remove unused parameter 'pos' from _match()
Yuya Nishihara <yuya@tcha.org>
parents:
25801
diff
changeset
|
37 def _match(self, m): |
11274 | 38 'make sure the tokenizer matches an end condition' |
39 if self.current[0] != m: | |
14701
4b93bd041772
parsers: fix localization markup of parser errors
Mads Kiilerich <mads@kiilerich.com>
parents:
13665
diff
changeset
|
40 raise error.ParseError(_("unexpected token: %s") % self.current[0], |
11305
d4cafcb63f77
cleanups: undefined variables
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
11289
diff
changeset
|
41 self.current[2]) |
11274 | 42 self._advance() |
25803
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
43 def _parseoperand(self, bind, m=None): |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
44 'gather right-hand-side operand until an end condition or binding met' |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
45 if m and self.current[0] == m: |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
46 expr = None |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
47 else: |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
48 expr = self._parse(bind) |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
49 if m: |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
50 self._match(m) |
b3004d273874
parser: factor out function that parses right-hand side of prefix/infix ops
Yuya Nishihara <yuya@tcha.org>
parents:
25802
diff
changeset
|
51 return expr |
11274 | 52 def _parse(self, bind=0): |
11289
4215ce511134
revset: raise ParseError exceptions
Matt Mackall <mpm@selenic.com>
parents:
11278
diff
changeset
|
53 token, value, pos = self._advance() |
25816
43a8a87fc175
parser: resolve ambiguity where both prefix and primary actions are defined
Yuya Nishihara <yuya@tcha.org>
parents:
25815
diff
changeset
|
54 # handle prefix rules on current token, take as primary if unambiguous |
25815
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
55 primary, prefix = self._elements[token][1:3] |
25816
43a8a87fc175
parser: resolve ambiguity where both prefix and primary actions are defined
Yuya Nishihara <yuya@tcha.org>
parents:
25815
diff
changeset
|
56 if primary and not (prefix and self._hasnewterm()): |
25815
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
57 expr = (primary, value) |
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
58 elif prefix: |
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
59 expr = (prefix[0], self._parseoperand(*prefix[1:])) |
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
60 else: |
14701
4b93bd041772
parsers: fix localization markup of parser errors
Mads Kiilerich <mads@kiilerich.com>
parents:
13665
diff
changeset
|
61 raise error.ParseError(_("not a prefix: %s") % token, pos) |
11274 | 62 # gather tokens until we meet a lower binding strength |
63 while bind < self._elements[self.current[0]][0]: | |
11289
4215ce511134
revset: raise ParseError exceptions
Matt Mackall <mpm@selenic.com>
parents:
11278
diff
changeset
|
64 token, value, pos = self._advance() |
25817
42ac9d1d1572
parser: reorder infix/suffix handling to be similar to prefix/primary flow
Yuya Nishihara <yuya@tcha.org>
parents:
25816
diff
changeset
|
65 # handle infix rules, take as suffix if unambiguous |
25815
e71e5629e006
parser: separate actions for primary expression and prefix operator
Yuya Nishihara <yuya@tcha.org>
parents:
25804
diff
changeset
|
66 infix, suffix = self._elements[token][3:] |
25818
455190fb4e51
parser: take suffix action if no infix action is defined
Yuya Nishihara <yuya@tcha.org>
parents:
25817
diff
changeset
|
67 if suffix and not (infix and self._hasnewterm()): |
11278
7df88cdf47fd
revset: add support for prefix and suffix versions of : and ::
Matt Mackall <mpm@selenic.com>
parents:
11274
diff
changeset
|
68 expr = (suffix[0], expr) |
25817
42ac9d1d1572
parser: reorder infix/suffix handling to be similar to prefix/primary flow
Yuya Nishihara <yuya@tcha.org>
parents:
25816
diff
changeset
|
69 elif infix: |
42ac9d1d1572
parser: reorder infix/suffix handling to be similar to prefix/primary flow
Yuya Nishihara <yuya@tcha.org>
parents:
25816
diff
changeset
|
70 expr = (infix[0], expr, self._parseoperand(*infix[1:])) |
11274 | 71 else: |
25817
42ac9d1d1572
parser: reorder infix/suffix handling to be similar to prefix/primary flow
Yuya Nishihara <yuya@tcha.org>
parents:
25816
diff
changeset
|
72 raise error.ParseError(_("not an infix: %s") % token, pos) |
11274 | 73 return expr |
25654
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
74 def parse(self, tokeniter): |
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
75 'generate a parse tree from tokens' |
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
76 self._iter = tokeniter |
13176
895f54a79c6e
templater: use the parser.py parser to extend the templater syntax
Matt Mackall <mpm@selenic.com>
parents:
11449
diff
changeset
|
77 self._advance() |
13665
e798e430c5e5
revset: report a parse error if a revset is not parsed completely (issue2654)
Bernhard Leiner <bleiner@gmail.com>
parents:
13176
diff
changeset
|
78 res = self._parse() |
e798e430c5e5
revset: report a parse error if a revset is not parsed completely (issue2654)
Bernhard Leiner <bleiner@gmail.com>
parents:
13176
diff
changeset
|
79 token, value, pos = self.current |
e798e430c5e5
revset: report a parse error if a revset is not parsed completely (issue2654)
Bernhard Leiner <bleiner@gmail.com>
parents:
13176
diff
changeset
|
80 return res, pos |
11274 | 81 def eval(self, tree): |
82 'recursively evaluate a parse tree using node methods' | |
83 if not isinstance(tree, tuple): | |
84 return tree | |
85 return self._methods[tree[0]](*[self.eval(t) for t in tree[1:]]) | |
25654
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
86 def __call__(self, tokeniter): |
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
87 'parse tokens into a parse tree and evaluate if methods given' |
af329a84310c
parser: accept iterator of tokens instead of tokenizer function and program
Yuya Nishihara <yuya@tcha.org>
parents:
25306
diff
changeset
|
88 t = self.parse(tokeniter) |
11274 | 89 if self._methods: |
90 return self.eval(t) | |
91 return t | |
25253
3f1a9b44b8c2
parser: move prettyformat() function from revset module
Yuya Nishihara <yuya@tcha.org>
parents:
25171
diff
changeset
|
92 |
25705
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
93 def buildargsdict(trees, funcname, keys, keyvaluenode, keynode): |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
94 """Build dict from list containing positional and keyword arguments |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
95 |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
96 Invalid keywords or too many positional arguments are rejected, but |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
97 missing arguments are just omitted. |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
98 """ |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
99 if len(trees) > len(keys): |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
100 raise error.ParseError(_("%(func)s takes at most %(nargs)d arguments") |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
101 % {'func': funcname, 'nargs': len(keys)}) |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
102 args = {} |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
103 # consume positional arguments |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
104 for k, x in zip(keys, trees): |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
105 if x[0] == keyvaluenode: |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
106 break |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
107 args[k] = x |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
108 # remainder should be keyword arguments |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
109 for x in trees[len(args):]: |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
110 if x[0] != keyvaluenode or x[1][0] != keynode: |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
111 raise error.ParseError(_("%(func)s got an invalid argument") |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
112 % {'func': funcname}) |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
113 k = x[1][1] |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
114 if k not in keys: |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
115 raise error.ParseError(_("%(func)s got an unexpected keyword " |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
116 "argument '%(key)s'") |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
117 % {'func': funcname, 'key': k}) |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
118 if k in args: |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
119 raise error.ParseError(_("%(func)s got multiple values for keyword " |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
120 "argument '%(key)s'") |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
121 % {'func': funcname, 'key': k}) |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
122 args[k] = x[2] |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
123 return args |
48919d246a47
revset: add function to build dict of positional and keyword arguments
Yuya Nishihara <yuya@tcha.org>
parents:
25655
diff
changeset
|
124 |
25254
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
125 def _prettyformat(tree, leafnodes, level, lines): |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
126 if not isinstance(tree, tuple) or tree[0] in leafnodes: |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
127 lines.append((level, str(tree))) |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
128 else: |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
129 lines.append((level, '(%s' % tree[0])) |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
130 for s in tree[1:]: |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
131 _prettyformat(s, leafnodes, level + 1, lines) |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
132 lines[-1:] = [(lines[-1][0], lines[-1][1] + ')')] |
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
133 |
25253
3f1a9b44b8c2
parser: move prettyformat() function from revset module
Yuya Nishihara <yuya@tcha.org>
parents:
25171
diff
changeset
|
134 def prettyformat(tree, leafnodes): |
3f1a9b44b8c2
parser: move prettyformat() function from revset module
Yuya Nishihara <yuya@tcha.org>
parents:
25171
diff
changeset
|
135 lines = [] |
25254
060bdfef2517
parser: extract closure of prettyformat() to a top-level function
Yuya Nishihara <yuya@tcha.org>
parents:
25253
diff
changeset
|
136 _prettyformat(tree, leafnodes, 0, lines) |
25253
3f1a9b44b8c2
parser: move prettyformat() function from revset module
Yuya Nishihara <yuya@tcha.org>
parents:
25171
diff
changeset
|
137 output = '\n'.join((' ' * l + s) for l, s in lines) |
3f1a9b44b8c2
parser: move prettyformat() function from revset module
Yuya Nishihara <yuya@tcha.org>
parents:
25171
diff
changeset
|
138 return output |
25306
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
139 |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
140 def simplifyinfixops(tree, targetnodes): |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
141 """Flatten chained infix operations to reduce usage of Python stack |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
142 |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
143 >>> def f(tree): |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
144 ... print prettyformat(simplifyinfixops(tree, ('or',)), ('symbol',)) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
145 >>> f(('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
146 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
147 ... ('symbol', '1'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
148 ... ('symbol', '2')), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
149 ... ('symbol', '3'))) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
150 (or |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
151 ('symbol', '1') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
152 ('symbol', '2') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
153 ('symbol', '3')) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
154 >>> f(('func', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
155 ... ('symbol', 'p1'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
156 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
157 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
158 ... ('func', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
159 ... ('symbol', 'sort'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
160 ... ('list', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
161 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
162 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
163 ... ('symbol', '1'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
164 ... ('symbol', '2')), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
165 ... ('symbol', '3')), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
166 ... ('negate', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
167 ... ('symbol', 'rev')))), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
168 ... ('and', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
169 ... ('symbol', '4'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
170 ... ('group', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
171 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
172 ... ('or', |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
173 ... ('symbol', '5'), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
174 ... ('symbol', '6')), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
175 ... ('symbol', '7'))))), |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
176 ... ('symbol', '8')))) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
177 (func |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
178 ('symbol', 'p1') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
179 (or |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
180 (func |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
181 ('symbol', 'sort') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
182 (list |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
183 (or |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
184 ('symbol', '1') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
185 ('symbol', '2') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
186 ('symbol', '3')) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
187 (negate |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
188 ('symbol', 'rev')))) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
189 (and |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
190 ('symbol', '4') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
191 (group |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
192 (or |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
193 ('symbol', '5') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
194 ('symbol', '6') |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
195 ('symbol', '7')))) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
196 ('symbol', '8'))) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
197 """ |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
198 if not isinstance(tree, tuple): |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
199 return tree |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
200 op = tree[0] |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
201 if op not in targetnodes: |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
202 return (op,) + tuple(simplifyinfixops(x, targetnodes) for x in tree[1:]) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
203 |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
204 # walk down left nodes taking each right node. no recursion to left nodes |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
205 # because infix operators are left-associative, i.e. left tree is deep. |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
206 # e.g. '1 + 2 + 3' -> (+ (+ 1 2) 3) -> (+ 1 2 3) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
207 simplified = [] |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
208 x = tree |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
209 while x[0] == op: |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
210 l, r = x[1:] |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
211 simplified.append(simplifyinfixops(r, targetnodes)) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
212 x = l |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
213 simplified.append(simplifyinfixops(x, targetnodes)) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
214 simplified.append(op) |
c87b05925054
parser: add helper to reduce nesting of chained infix operations
Yuya Nishihara <yuya@tcha.org>
parents:
25254
diff
changeset
|
215 return tuple(reversed(simplified)) |