Mercurial > public > mercurial-scm > hg

comparison contrib/byteify-strings.py @ 42676:b9a200477edf

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
byteify-strings: add support for ignore comments Our simple token analysis is sometimes not clever enough, we need to be able to turn off our script for parts of the code. This change introduces three special comments: - `#no-py3-transform` to tell `byteify-strings` ignore the next line - `#py3-transform: off` to ignore everything until the end of the file - `#py3-transform: on` to stop ignoring The last two can be particularly useful within Python 2/3 compatibility files.
author Rapha?l Gom?s <rgomes@octobus.net>
date Fri, 02 Aug 2019 09:55:32 +0200
parents e9592e113c31
children c9fd8163131f
comparison
equal deleted inserted replaced
42675:e9592e113c31 42676:b9a200477edf
93 break 93 break
94 94
95 coldelta = 0 # column increment for new opening parens 95 coldelta = 0 # column increment for new opening parens
96 coloffset = -1 # column offset for the current line (-1: TBD) 96 coloffset = -1 # column offset for the current line (-1: TBD)
97 parens = [(0, 0, 0)] # stack of (line, end-column, column-offset) 97 parens = [(0, 0, 0)] # stack of (line, end-column, column-offset)
98 ignorenextline = False # don't transform the next line
99 insideignoreblock = False # don't transform until turned off
98 for i, t in enumerate(tokens): 100 for i, t in enumerate(tokens):
99 # Compute the column offset for the current line, such that 101 # Compute the column offset for the current line, such that
100 # the current line will be aligned to the last opening paren 102 # the current line will be aligned to the last opening paren
101 # as before. 103 # as before.
102 if coloffset < 0: 104 if coloffset < 0:
111 # Reset per-line attributes at EOL. 113 # Reset per-line attributes at EOL.
112 if t.type in (token.NEWLINE, tokenize.NL): 114 if t.type in (token.NEWLINE, tokenize.NL):
113 yield adjusttokenpos(t, coloffset) 115 yield adjusttokenpos(t, coloffset)
114 coldelta = 0 116 coldelta = 0
115 coloffset = -1 117 coloffset = -1
118 if not insideignoreblock:
119 ignorenextline = (
120 tokens[i - 1].type == token.COMMENT
121 and tokens[i - 1].string == "#no-py3-transform"
122 )
123 continue
124
125 if t.type == token.COMMENT:
126 if t.string == "#py3-transform: off":
127 insideignoreblock = True
128 if t.string == "#py3-transform: on":
129 insideignoreblock = False
130
131 if ignorenextline or insideignoreblock:
132 yield adjusttokenpos(t, coloffset)
116 continue 133 continue
117 134
118 # Remember the last paren position. 135 # Remember the last paren position.
119 if _isop(i, '(', '[', '{'): 136 if _isop(i, '(', '[', '{'):
120 parens.append(t.end + (coloffset + coldelta,)) 137 parens.append(t.end + (coloffset + coldelta,))