mercurial-scm/hg: hgext/fix.py annotate

annotate hgext/fix.py @ 48178:f12a19d03d2c

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280

author	Danny Hooper <hooper@google.com>
date	Thu, 02 Sep 2021 14:08:45 -0700
parents	5ced12cfa41b
children	2f7caef017d9

rev	line source
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	1 # fix - rewrite file content in changesets and working copy
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	2 #
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	3 # Copyright 2018 Google LLC.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	4 #
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	7 """rewrite file content in changesets or working copy (EXPERIMENTAL)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	8
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	9 Provides a command that runs configured tools on the contents of modified files,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	10 writing back any fixes to the working copy or replacing changesets.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	11
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	12 Here is an example configuration that causes :hg:`fix` to apply automatic
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	13 formatting fixes to modified lines in C++ code::
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	14
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	15 [fix]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	16 clang-format:command=clang-format --assume-filename={rootpath}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	17 clang-format:linerange=--lines={first}:{last}
40533 2ecf5c24d0cd fix: rename :fileset subconfig to :pattern Danny Hooper <hooper@google.com> parents: 40532 diff changeset	18 clang-format:pattern=set:.cpp or .hpp
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	19
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	20 The :command suboption forms the first part of the shell command that will be
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	21 used to fix a file. The content of the file is passed on standard input, and the
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	22 fixed file content is expected on standard output. Any output on standard error
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	23 will be displayed as a warning. If the exit status is not zero, the file will
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	24 not be affected. A placeholder warning is displayed if there is a non-zero exit
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	25 status but no standard error output. Some values may be substituted into the
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	26 command::
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	27
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	28 {rootpath} The path of the file being fixed, relative to the repo root
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	29 {basename} The name of the file being fixed, without the directory path
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	30
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	31 If the :linerange suboption is set, the tool will only be run if there are
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	32 changed lines in a file. The value of this suboption is appended to the shell
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	33 command once for every range of changed lines in the file. Some values may be
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	34 substituted into the command::
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	35
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	36 {first} The 1-based line number of the first line in the modified range
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	37 {last} The 1-based line number of the last line in the modified range
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	38
42756 ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	39 Deleted sections of a file will be ignored by :linerange, because there is no
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	40 corresponding line range in the version being fixed.
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	41
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	42 By default, tools that set :linerange will only be executed if there is at least
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	43 one changed line range. This is meant to prevent accidents like running a code
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	44 formatter in such a way that it unexpectedly reformats the whole file. If such a
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	45 tool needs to operate on unchanged files, it should set the :skipclean suboption
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	46 to false.
ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	47
40533 2ecf5c24d0cd fix: rename :fileset subconfig to :pattern Danny Hooper <hooper@google.com> parents: 40532 diff changeset	48 The :pattern suboption determines which files will be passed through each
43227 f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	49 configured tool. See :hg:`help patterns` for possible values. However, all
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	50 patterns are relative to the repo root, even if that text says they are relative
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	51 to the current working directory. If there are file arguments to :hg:`fix`, the
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	52 intersection of these patterns is used.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	53
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	54 There is also a configurable limit for the maximum size of file that will be
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	55 processed by :hg:`fix`::
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	56
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	57 [fix]
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	58 maxfilesize = 2MB
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	59
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	60 Normally, execution of configured tools will continue after a failure (indicated
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	61 by a non-zero exit status). It can also be configured to abort after the first
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	62 such failure, so that no files will be affected if any tool fails. This abort
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	63 will also cause :hg:`fix` to exit with a non-zero status::
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	64
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	65 [fix]
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	66 failure = abort
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	67
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	68 When multiple tools are configured to affect a file, they execute in an order
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	69 defined by the :priority suboption. The priority suboption has a default value
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	70 of zero for each tool. Tools are executed in order of descending priority. The
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	71 execution order of tools with equal priority is unspecified. For example, you
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	72 could use the 'sort' and 'head' utilities to keep only the 10 smallest numbers
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	73 in a text file by ensuring that 'sort' runs before 'head'::
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	74
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	75 [fix]
41126 d8f5c615e811 tests: use more portable flags in test-fix.t Danny Hooper <hooper@google.com> parents: 40570 diff changeset	76 sort:command = sort -n
d8f5c615e811 tests: use more portable flags in test-fix.t Danny Hooper <hooper@google.com> parents: 40570 diff changeset	77 head:command = head -n 10
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	78 sort:pattern = numbers.txt
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	79 head:pattern = numbers.txt
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	80 sort:priority = 2
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	81 head:priority = 1
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	82
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	83 To account for changes made by each tool, the line numbers used for incremental
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	84 formatting are recomputed before executing the next tool. So, each tool may see
b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	85 different values for the arguments added by the :linerange suboption.
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	86
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	87 Each fixer tool is allowed to return some metadata in addition to the fixed file
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	88 content. The metadata must be placed before the file content on stdout,
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	89 separated from the file content by a zero byte. The metadata is parsed as a JSON
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	90 value (so, it should be UTF-8 encoded and contain no zero bytes). A fixer tool
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	91 is expected to produce this metadata encoding if and only if the :metadata
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	92 suboption is true::
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	93
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	94 [fix]
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	95 tool:command = tool --prepend-json-metadata
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	96 tool:metadata = true
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	97
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	98 The metadata values are passed to hooks, which can be used to print summaries or
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	99 perform other post-fixing work. The supported hooks are::
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	100
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	101 "postfixfile"
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	102 Run once for each file in each revision where any fixer tools made changes
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	103 to the file content. Provides "$HG_REV" and "$HG_PATH" to identify the file,
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	104 and "$HG_METADATA" with a map of fixer names to metadata values from fixer
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	105 tools that affected the file. Fixer tools that didn't affect the file have a
44048 61881b170140 fix: fix grammar/typos in hg help -e fix timeless <timeless@mozdev.org> parents: 43380 diff changeset	106 value of None. Only fixer tools that executed are present in the metadata.
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	107
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	108 "postfix"
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	109 Run once after all files and revisions have been handled. Provides
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	110 "$HG_REPLACEMENTS" with information about what revisions were created and
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	111 made obsolete. Provides a boolean "$HG_WDIRWRITTEN" to indicate whether any
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	112 files in the working copy were updated. Provides a list "$HG_METADATA"
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	113 mapping fixer tool names to lists of metadata values returned from
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	114 executions that modified a file. This aggregates the same metadata
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	115 previously passed to the "postfixfile" hook.
42673 74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	116
44048 61881b170140 fix: fix grammar/typos in hg help -e fix timeless <timeless@mozdev.org> parents: 43380 diff changeset	117 Fixer tools are run in the repository's root directory. This allows them to read
42673 74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	118 configuration files from the working copy, or even write to the working copy.
74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	119 The working copy is not updated to match the revision being fixed. In fact,
74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	120 several revisions may be fixed in parallel. Writes to the working copy are not
74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	121 amended into the revision being fixed; fixer tools should always write fixed
74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	122 file content back to stdout as documented above.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	123 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	124
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	125 from __future__ import absolute_import
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	126
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	127 import collections
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	128 import itertools
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	129 import os
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	130 import re
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	131 import subprocess
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	132
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	133 from mercurial.i18n import _
46839 eb2a6f66c463 fix: merge imports Joerg Sonnenberger <joerg@bec.de> parents: 46361 diff changeset	134 from mercurial.node import (
47766 3feda1e779d4 fix: rewrite writeworkingdir() to explicitly not work with merges Martin von Zweigbergk <martinvonz@google.com> parents: 47735 diff changeset	135 nullid,
46839 eb2a6f66c463 fix: merge imports Joerg Sonnenberger <joerg@bec.de> parents: 46361 diff changeset	136 nullrev,
eb2a6f66c463 fix: merge imports Joerg Sonnenberger <joerg@bec.de> parents: 46361 diff changeset	137 wdirrev,
eb2a6f66c463 fix: merge imports Joerg Sonnenberger <joerg@bec.de> parents: 46361 diff changeset	138 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	139
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	140 from mercurial.utils import procutil
39826 c31ce080eb75 py3: convert arguments, cwd and env to native strings when spawning subprocess Matt Harbison <matt_harbison@yahoo.com> parents: 38967 diff changeset	141
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	142 from mercurial import (
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	143 cmdutil,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	144 context,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	145 copies,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	146 error,
48116 5ced12cfa41b errors: raise InputError on bad revset to revrange() iff provided by the user Martin von Zweigbergk <martinvonz@google.com> parents: 47924 diff changeset	147 logcmdutil,
43227 f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	148 match as matchmod,
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	149 mdiff,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	150 merge,
44856 b7808443ed6a mergestate: split out merge state handling code from main merge module Augie Fackler <augie@google.com> parents: 44575 diff changeset	151 mergestate as mergestatemod,
47797 e69c82bf3a01 fix: use obsolete.isenabled() to check for experimental.allowdivergence Anton Shestakov <av6@dwimlabs.net> parents: 47767 diff changeset	152 obsolete,
37618 1edf3738e000 fix: port most of the way to python 3 Augie Fackler <augie@google.com> parents: 37595 diff changeset	153 pycompat,
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	154 registrar,
43936 699d6be3820a fix: use rewriteutil.precheck() instead of reimplementing it Martin von Zweigbergk <martinvonz@google.com> parents: 43897 diff changeset	155 rewriteutil,
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	156 scmutil,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	157 util,
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	158 worker,
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	159 )
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	160
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	161 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	162 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	163 # be specifying the version(s) of Mercurial they are tested with, or
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	164 # leave the attribute unspecified.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	165 testedwith = b'ships-with-hg-core'
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	166
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	167 cmdtable = {}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	168 command = registrar.command(cmdtable)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	169
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	170 configtable = {}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	171 configitem = registrar.configitem(configtable)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	172
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	173 # Register the suboptions allowed for each configured fixer, and default values.
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	174 FIXER_ATTRS = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	175 b'command': None,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	176 b'linerange': None,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	177 b'pattern': None,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	178 b'priority': 0,
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	179 b'metadata': False,
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	180 b'skipclean': True,
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	181 b'enabled': True,
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	182 }
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	183
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	184 for key, default in FIXER_ATTRS.items():
43217 5cb3e6f4e069 fix: fix registration of config item defaults Martin von Zweigbergk <martinvonz@google.com> parents: 43193 diff changeset	185 configitem(b'fix', b'.*:%s$' % key, default=default, generic=True)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	186
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	187 # A good default size allows most source code files to be fixed, but avoids
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	188 # letting fixer tools choke on huge inputs, which could be surprising to the
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	189 # user.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	190 configitem(b'fix', b'maxfilesize', default=b'2MB')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	191
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	192 # Allow fix commands to exit non-zero if an executed fixer tool exits non-zero.
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	193 # This helps users do shell scripts that stop when a fixer tool signals a
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	194 # problem.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	195 configitem(b'fix', b'failure', default=b'continue')
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	196
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	197
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	198 def checktoolfailureaction(ui, message, hint=None):
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	199 """Abort with 'message' if fix.failure=abort"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	200 action = ui.config(b'fix', b'failure')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	201 if action not in (b'continue', b'abort'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	202 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	203 _(b'unknown fix.failure action: %s') % (action,),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	204 hint=_(b'use "continue" or "abort"'),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	205 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	206 if action == b'abort':
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	207 raise error.Abort(message, hint=hint)
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	208
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	209
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	210 allopt = (b'', b'all', False, _(b'fix all non-public non-obsolete revisions'))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	211 baseopt = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	212 b'',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	213 b'base',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	214 [],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	215 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	216 b'revisions to diff against (overrides automatic '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	217 b'selection, and applies to every revision being '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	218 b'fixed)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	219 ),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	220 _(b'REV'),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	221 )
44575 a6ef1e8e2f6d fix: mark -r as advanced Martin von Zweigbergk <martinvonz@google.com> parents: 44574 diff changeset	222 revopt = (b'r', b'rev', [], _(b'revisions to fix (ADVANCED)'), _(b'REV'))
44574 5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	223 sourceopt = (
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	224 b's',
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	225 b'source',
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	226 [],
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	227 _(b'fix the specified revisions and their descendants'),
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	228 _(b'REV'),
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	229 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	230 wdiropt = (b'w', b'working-dir', False, _(b'fix the working directory'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	231 wholeopt = (b'', b'whole', False, _(b'always fix every line of a file'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	232 usage = _(b'[OPTION]... [FILE]...')
38949 b0c591950e51 fix: pull out flag definitions to make them re-usable from extensions Danny Hooper <hooper@google.com> parents: 38860 diff changeset	233
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	234
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	235 @command(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	236 b'fix',
44574 5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	237 [allopt, baseopt, revopt, sourceopt, wdiropt, wholeopt],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	238 usage,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	239 helpcategory=command.CATEGORY_FILE_CONTENTS,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	240 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	241 def fix(ui, repo, pats, *opts):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	242 """rewrite file content in changesets or working directory
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	243
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	244 Runs any configured tools to fix the content of files. Only affects files
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	245 with changes, unless file arguments are provided. Only affects changed lines
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	246 of files, unless the --whole flag is used. Some tools may always affect the
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	247 whole file regardless of --whole.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	248
45218 3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	249 If --working-dir is used, files with uncommitted changes in the working copy
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	250 will be fixed. Note that no backup are made.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	251
45218 3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	252 If revisions are specified with --source, those revisions and their
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	253 descendants will be checked, and they may be replaced with new revisions
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	254 that have fixed file content. By automatically including the descendants,
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	255 no merging, rebasing, or evolution will be required. If an ancestor of the
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	256 working copy is included, then the working copy itself will also be fixed,
3ea3b85df03f fix: update documentation to reflect preference for --source over --rev Martin von Zweigbergk <martinvonz@google.com> parents: 45074 diff changeset	257 and the working copy will be updated to the fixed parent.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	258
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	259 When determining what lines of each file to fix at each revision, the whole
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	260 set of revisions being fixed is considered, so that fixes to earlier
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	261 revisions are not forgotten in later ones. The --base flag can be used to
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	262 override this default behavior, though it is not usually desirable to do so.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	263 """
37618 1edf3738e000 fix: port most of the way to python 3 Augie Fackler <augie@google.com> parents: 37595 diff changeset	264 opts = pycompat.byteskwargs(opts)
44574 5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	265 cmdutil.check_at_most_one_arg(opts, b'all', b'source', b'rev')
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	266 cmdutil.check_incompatible_arguments(
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	267 opts, b'working_dir', [b'all', b'source']
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	268 )
44573 9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	269
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	270 with repo.wlock(), repo.lock(), repo.transaction(b'fix'):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	271 revstofix = getrevstofix(ui, repo, opts)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	272 basectxs = getbasectxs(repo, opts, revstofix)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	273 workqueue, numitems = getworkqueue(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	274 ui, repo, pats, opts, revstofix, basectxs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	275 )
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	276 basepaths = getbasepaths(repo, opts, workqueue, basectxs)
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	277 fixers = getfixers(ui)
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	278
45074 797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	279 # Rather than letting each worker independently fetch the files
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	280 # (which also would add complications for shared/keepalive
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	281 # connections), prefetch them all first.
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	282 _prefetchfiles(repo, workqueue, basepaths)
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	283
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	284 # There are no data dependencies between the workers fixing each file
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	285 # revision, so we can use all available parallelism.
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	286 def getfixes(items):
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	287 for srcrev, path, dstrevs in items:
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	288 ctx = repo[srcrev]
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	289 olddata = ctx[path].data()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	290 metadata, newdata = fixfile(
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	291 ui,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	292 repo,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	293 opts,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	294 fixers,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	295 ctx,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	296 path,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	297 basepaths,
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	298 basectxs[srcrev],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	299 )
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	300 # We ungroup the work items now, because the code that consumes
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	301 # these results has to handle each dstrev separately, and in
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	302 # topological order. Because these are handled in topological
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	303 # order, it's important that we pass around references to
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	304 # "newdata" instead of copying it. Otherwise, we would be
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	305 # keeping more copies of file content in memory at a time than
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	306 # if we hadn't bothered to group/deduplicate the work items.
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	307 data = newdata if newdata != olddata else None
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	308 for dstrev in dstrevs:
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	309 yield (dstrev, path, metadata, data)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	310
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	311 results = worker.worker(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	312 ui, 1.0, getfixes, tuple(), workqueue, threadsafe=False
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	313 )
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	314
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	315 # We have to hold on to the data for each successor revision in memory
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	316 # until all its parents are committed. We ensure this by committing and
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	317 # freeing memory for the revisions in some topological order. This
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	318 # leaves a little bit of memory efficiency on the table, but also makes
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	319 # the tests deterministic. It might also be considered a feature since
5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	320 # it makes the results more easily reproducible.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	321 filedata = collections.defaultdict(dict)
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	322 aggregatemetadata = collections.defaultdict(list)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	323 replacements = {}
38950 35bc4b6e132d fix: correctly set wdirwritten given that the dict item is deleted Danny Hooper <hooper@google.com> parents: 38949 diff changeset	324 wdirwritten = False
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	325 commitorder = sorted(revstofix, reverse=True)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	326 with ui.makeprogress(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	327 topic=_(b'fixing'), unit=_(b'files'), total=sum(numitems.values())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	328 ) as progress:
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	329 for rev, path, filerevmetadata, newdata in results:
38537 a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	330 progress.increment(item=path)
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	331 for fixername, fixermetadata in filerevmetadata.items():
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	332 aggregatemetadata[fixername].append(fixermetadata)
38537 a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	333 if newdata is not None:
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	334 filedata[rev][path] = newdata
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	335 hookargs = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	336 b'rev': rev,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	337 b'path': path,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	338 b'metadata': filerevmetadata,
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	339 }
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	340 repo.hook(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	341 b'postfixfile',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	342 throw=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	343 **pycompat.strkwargs(hookargs)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	344 )
38537 a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	345 numitems[rev] -= 1
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	346 # Apply the fixes for this and any other revisions that are
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	347 # ready and sitting at the front of the queue. Using a loop here
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	348 # prevents the queue from being blocked by the first revision to
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	349 # be ready out of order.
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	350 while commitorder and not numitems[commitorder[-1]]:
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	351 rev = commitorder.pop()
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	352 ctx = repo[rev]
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	353 if rev == wdirrev:
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	354 writeworkingdir(repo, ctx, filedata[rev], replacements)
38950 35bc4b6e132d fix: correctly set wdirwritten given that the dict item is deleted Danny Hooper <hooper@google.com> parents: 38949 diff changeset	355 wdirwritten = bool(filedata[rev])
38537 a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	356 else:
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	357 replacerev(ui, repo, ctx, filedata[rev], replacements)
a3be09e277e9 fix: add progress bar for number of file revisions processed Danny Hooper <hooper@google.com> parents: 38536 diff changeset	358 del filedata[rev]
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	359
38950 35bc4b6e132d fix: correctly set wdirwritten given that the dict item is deleted Danny Hooper <hooper@google.com> parents: 38949 diff changeset	360 cleanup(repo, replacements, wdirwritten)
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	361 hookargs = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	362 b'replacements': replacements,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 b'wdirwritten': wdirwritten,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	364 b'metadata': aggregatemetadata,
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	365 }
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	366 repo.hook(b'postfix', throw=True, **pycompat.strkwargs(hookargs))
38811 64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	367
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	368
38811 64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	369 def cleanup(repo, replacements, wdirwritten):
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	370 """Calls scmutil.cleanupnodes() with the given replacements.
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	371
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	372 "replacements" is a dict from nodeid to nodeid, with one key and one value
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	373 for every revision that was affected by fixing. This is slightly different
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	374 from cleanupnodes().
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	375
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	376 "wdirwritten" is a bool which tells whether the working copy was affected by
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	377 fixing, since it has no entry in "replacements".
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	378
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	379 Useful as a hook point for extending "hg fix" with output summarizing the
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	380 effects of the command, though we choose not to output anything here.
64535d43c103 fix: add a monkey-patchable point after all new revisions have been committed Danny Hooper <hooper@google.com> parents: 38770 diff changeset	381 """
43105 649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/ Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	382 replacements = {
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/ Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	383 prec: [succ] for prec, succ in pycompat.iteritems(replacements)
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/ Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	384 }
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	385 scmutil.cleanupnodes(repo, replacements, b'fix', fixphase=True)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	386
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	387
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	388 def getworkqueue(ui, repo, pats, opts, revstofix, basectxs):
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	389 """Constructs a list of files to fix and which revisions each fix applies to
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	390
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	391 To avoid duplicating work, there is usually only one work item for each file
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	392 revision that might need to be fixed. There can be multiple work items per
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	393 file revision if the same file needs to be fixed in multiple changesets with
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	394 different baserevs. Each work item also contains a list of changesets where
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	395 the file's data should be replaced with the fixed data. The work items for
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	396 earlier changesets come earlier in the work queue, to improve pipelining by
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	397 allowing the first changeset to be replaced while fixes are still being
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	398 computed for later changesets.
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	399
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	400 Also returned is a map from changesets to the count of work items that might
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	401 affect each changeset. This is used later to count when all of a changeset's
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	402 work items have been finished, without having to inspect the remaining work
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	403 queue in each worker subprocess.
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	404
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	405 The example work item (1, "foo/bar.txt", (1, 2, 3)) means that the data of
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	406 bar.txt should be read from revision 1, then fixed, and written back to
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	407 revisions 1, 2 and 3. Revision 1 is called the "srcrev" and the list of
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	408 revisions is called the "dstrevs". In practice the srcrev is always one of
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	409 the dstrevs, and we make that choice when constructing the work item so that
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	410 the choice can't be made inconsistently later on. The dstrevs should all
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	411 have the same file revision for the given path, so the choice of srcrev is
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	412 arbitrary. The wdirrev can be a dstrev and a srcrev.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	413 """
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	414 dstrevmap = collections.defaultdict(list)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	415 numitems = collections.defaultdict(int)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	416 maxfilesize = ui.configbytes(b'fix', b'maxfilesize')
38536 5ffe2041d427 fix: use a worker pool to parallelize running tools Danny Hooper <hooper@google.com> parents: 38423 diff changeset	417 for rev in sorted(revstofix):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	418 fixctx = repo[rev]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	419 match = scmutil.match(fixctx, pats, opts)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	420 for path in sorted(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	421 pathstofix(ui, repo, pats, opts, match, basectxs[rev], fixctx)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	422 ):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	423 fctx = fixctx[path]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	424 if fctx.islink():
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	425 continue
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	426 if fctx.size() > maxfilesize:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	427 ui.warn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	428 _(b'ignoring file larger than %s: %s\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	429 % (util.bytecount(maxfilesize), path)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	430 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	431 continue
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	432 baserevs = tuple(ctx.rev() for ctx in basectxs[rev])
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	433 dstrevmap[(fctx.filerev(), baserevs, path)].append(rev)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	434 numitems[rev] += 1
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	435 workqueue = [
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	436 (min(dstrevs), path, dstrevs)
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	437 for (filerev, baserevs, path), dstrevs in dstrevmap.items()
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	438 ]
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	439 # Move work items for earlier changesets to the front of the queue, so we
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	440 # might be able to replace those changesets (in topological order) while
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	441 # we're still processing later work items. Note the min() in the previous
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	442 # expression, which means we don't need a custom comparator here. The path
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	443 # is also important in the sort order to make the output order stable. There
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	444 # are some situations where this doesn't help much, but some situations
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	445 # where it lets us buffer O(1) files instead of O(n) files.
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	446 workqueue.sort()
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	447 return workqueue, numitems
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	448
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	449
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	450 def getrevstofix(ui, repo, opts):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	451 """Returns the set of revision numbers that should be fixed"""
44573 9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	452 if opts[b'all']:
9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	453 revs = repo.revs(b'(not public() and not obsolete()) or wdir()')
44574 5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	454 elif opts[b'source']:
48116 5ced12cfa41b errors: raise InputError on bad revset to revrange() iff provided by the user Martin von Zweigbergk <martinvonz@google.com> parents: 47924 diff changeset	455 source_revs = logcmdutil.revrange(repo, opts[b'source'])
45809 136a86327316 fix: don't include obsolete descendants with -s Martin von Zweigbergk <martinvonz@google.com> parents: 45807 diff changeset	456 revs = set(repo.revs(b'(%ld::) - obsolete()', source_revs))
44574 5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	457 if wdirrev in source_revs:
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	458 # `wdir()::` is currently empty, so manually add wdir
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	459 revs.add(wdirrev)
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	460 if repo[b'.'].rev() in revs:
5205b46bd887 fix: add a -s option to format a revision and its descendants Martin von Zweigbergk <martinvonz@google.com> parents: 44573 diff changeset	461 revs.add(wdirrev)
44573 9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	462 else:
48116 5ced12cfa41b errors: raise InputError on bad revset to revrange() iff provided by the user Martin von Zweigbergk <martinvonz@google.com> parents: 47924 diff changeset	463 revs = set(logcmdutil.revrange(repo, opts[b'rev']))
44573 9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	464 if opts.get(b'working_dir'):
9f5e94bbc606 fix: move handling of --all into getrevstofix() for consistency Martin von Zweigbergk <martinvonz@google.com> parents: 44561 diff changeset	465 revs.add(wdirrev)
45810 09914d934cf4 fix: only check for obsolete commits in the --rev case Martin von Zweigbergk <martinvonz@google.com> parents: 45809 diff changeset	466 for rev in revs:
09914d934cf4 fix: only check for obsolete commits in the --rev case Martin von Zweigbergk <martinvonz@google.com> parents: 45809 diff changeset	467 checkfixablectx(ui, repo, repo[rev])
44561 368f85c5dfc0 fix: refactor getrevstofix() to define revisions first, then validate them Martin von Zweigbergk <martinvonz@google.com> parents: 44560 diff changeset	468 # Allow fixing only wdir() even if there's an unfinished operation
368f85c5dfc0 fix: refactor getrevstofix() to define revisions first, then validate them Martin von Zweigbergk <martinvonz@google.com> parents: 44560 diff changeset	469 if not (len(revs) == 1 and wdirrev in revs):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	470 cmdutil.checkunfinished(repo)
43936 699d6be3820a fix: use rewriteutil.precheck() instead of reimplementing it Martin von Zweigbergk <martinvonz@google.com> parents: 43897 diff changeset	471 rewriteutil.precheck(repo, revs, b'fix')
46361 dfca84970da8 cleanup: use mergestate.unresolvedcount() instead of bool(list(unresolved())) Augie Fackler <augie@google.com> parents: 45946 diff changeset	472 if (
dfca84970da8 cleanup: use mergestate.unresolvedcount() instead of bool(list(unresolved())) Augie Fackler <augie@google.com> parents: 45946 diff changeset	473 wdirrev in revs
dfca84970da8 cleanup: use mergestate.unresolvedcount() instead of bool(list(unresolved())) Augie Fackler <augie@google.com> parents: 45946 diff changeset	474 and mergestatemod.mergestate.read(repo).unresolvedcount()
44856 b7808443ed6a mergestate: split out merge state handling code from main merge module Augie Fackler <augie@google.com> parents: 44575 diff changeset	475 ):
44561 368f85c5dfc0 fix: refactor getrevstofix() to define revisions first, then validate them Martin von Zweigbergk <martinvonz@google.com> parents: 44560 diff changeset	476 raise error.Abort(b'unresolved conflicts', hint=b"use 'hg resolve'")
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	477 if not revs:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	478 raise error.Abort(
45807 f90943d753ef fix: suggest --source instead of --rev on empty revset Martin von Zweigbergk <martinvonz@google.com> parents: 45713 diff changeset	479 b'no changesets specified', hint=b'use --source or --working-dir'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	480 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	481 return revs
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	482
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	483
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	484 def checkfixablectx(ui, repo, ctx):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	485 """Aborts if the revision shouldn't be replaced with a fixed one."""
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	486 if ctx.obsolete():
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	487 # It would be better to actually check if the revision has a successor.
47797 e69c82bf3a01 fix: use obsolete.isenabled() to check for experimental.allowdivergence Anton Shestakov <av6@dwimlabs.net> parents: 47767 diff changeset	488 if not obsolete.isenabled(repo, obsolete.allowdivergenceopt):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	489 raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	490 b'fixing obsolete revision could cause divergence'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	491 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	492
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	493
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	494 def pathstofix(ui, repo, pats, opts, match, basectxs, fixctx):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	495 """Returns the set of files that should be fixed in a context
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	496
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	497 The result depends on the base contexts; we include any file that has
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	498 changed relative to any of the base contexts. Base contexts should be
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	499 ancestors of the context being fixed.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	500 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	501 files = set()
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	502 for basectx in basectxs:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	503 stat = basectx.status(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	504 fixctx, match=match, listclean=bool(pats), listunknown=bool(pats)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	505 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	506 files.update(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	507 set(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	508 itertools.chain(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	509 stat.added, stat.modified, stat.clean, stat.unknown
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	510 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	511 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	512 )
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	513 return files
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	514
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	515
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	516 def lineranges(opts, path, basepaths, basectxs, fixctx, content2):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	517 """Returns the set of line ranges that should be fixed in a file
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	518
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	519 Of the form [(10, 20), (30, 40)].
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	520
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	521 This depends on the given base contexts; we must consider lines that have
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	522 changed versus any of the base contexts, and whether the file has been
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	523 renamed versus any of them.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	524
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	525 Another way to understand this is that we exclude line ranges that are
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	526 common to the file in all base contexts.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	527 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	528 if opts.get(b'whole'):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	529 # Return a range containing all lines. Rely on the diff implementation's
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	530 # idea of how many lines are in the file, instead of reimplementing it.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	531 return difflineranges(b'', content2)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	532
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	533 rangeslist = []
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	534 for basectx in basectxs:
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	535 basepath = basepaths.get((basectx.rev(), fixctx.rev(), path), path)
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	536
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	537 if basepath in basectx:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	538 content1 = basectx[basepath].data()
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	539 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	540 content1 = b''
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	541 rangeslist.extend(difflineranges(content1, content2))
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	542 return unionranges(rangeslist)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	543
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	544
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	545 def getbasepaths(repo, opts, workqueue, basectxs):
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	546 if opts.get(b'whole'):
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	547 # Base paths will never be fetched for line range determination.
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	548 return {}
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	549
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	550 basepaths = {}
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	551 for srcrev, path, _dstrevs in workqueue:
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	552 fixctx = repo[srcrev]
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	553 for basectx in basectxs[srcrev]:
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	554 basepath = copies.pathcopies(basectx, fixctx).get(path, path)
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	555 if basepath in basectx:
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	556 basepaths[(basectx.rev(), fixctx.rev(), path)] = basepath
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	557 return basepaths
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	558
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	559
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	560 def unionranges(rangeslist):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	561 """Return the union of some closed intervals
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	562
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	563 >>> unionranges([])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	564 []
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	565 >>> unionranges([(1, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	566 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	567 >>> unionranges([(1, 100), (1, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	568 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	569 >>> unionranges([(1, 100), (2, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	570 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	571 >>> unionranges([(1, 99), (1, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	572 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	573 >>> unionranges([(1, 100), (40, 60)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	574 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	575 >>> unionranges([(1, 49), (50, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	576 [(1, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	577 >>> unionranges([(1, 48), (50, 100)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	578 [(1, 48), (50, 100)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	579 >>> unionranges([(1, 2), (3, 4), (5, 6)])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	580 [(1, 6)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	581 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	582 rangeslist = sorted(set(rangeslist))
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	583 unioned = []
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	584 if rangeslist:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	585 unioned, rangeslist = [rangeslist[0]], rangeslist[1:]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	586 for a, b in rangeslist:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	587 c, d = unioned[-1]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	588 if a > d + 1:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	589 unioned.append((a, b))
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	590 else:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	591 unioned[-1] = (c, max(b, d))
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	592 return unioned
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	593
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	594
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	595 def difflineranges(content1, content2):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	596 """Return list of line number ranges in content2 that differ from content1.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	597
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	598 Line numbers are 1-based. The numbers are the first and last line contained
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	599 in the range. Single-line ranges have the same line number for the first and
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	600 last line. Excludes any empty ranges that result from lines that are only
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	601 present in content1. Relies on mdiff's idea of where the line endings are in
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	602 the string.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	603
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	604 >>> from mercurial import pycompat
893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	605 >>> lines = lambda s: b'\\n'.join([c for c in pycompat.iterbytestr(s)])
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	606 >>> difflineranges2 = lambda a, b: difflineranges(lines(a), lines(b))
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	607 >>> difflineranges2(b'', b'')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	608 []
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	609 >>> difflineranges2(b'a', b'')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	610 []
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	611 >>> difflineranges2(b'', b'A')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	612 [(1, 1)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	613 >>> difflineranges2(b'a', b'a')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	614 []
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	615 >>> difflineranges2(b'a', b'A')
893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	616 [(1, 1)]
893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	617 >>> difflineranges2(b'ab', b'')
893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	618 []
893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	619 >>> difflineranges2(b'', b'AB')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	620 [(1, 2)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	621 >>> difflineranges2(b'abc', b'ac')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	622 []
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	623 >>> difflineranges2(b'ab', b'aCb')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	624 [(2, 2)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	625 >>> difflineranges2(b'abc', b'aBc')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	626 [(2, 2)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	627 >>> difflineranges2(b'ab', b'AB')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	628 [(1, 2)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	629 >>> difflineranges2(b'abcde', b'aBcDe')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	630 [(2, 2), (4, 4)]
37213 893ff8c3bc57 py3: fix fix doctests to be bytes-safe Yuya Nishihara <yuya@tcha.org> parents: 37207 diff changeset	631 >>> difflineranges2(b'abcde', b'aBCDe')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	632 [(2, 4)]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	633 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	634 ranges = []
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	635 for lines, kind in mdiff.allblocks(content1, content2):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	636 firstline, lastline = lines[2:4]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	637 if kind == b'!' and firstline != lastline:
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	638 ranges.append((firstline + 1, lastline))
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	639 return ranges
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	640
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	641
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	642 def getbasectxs(repo, opts, revstofix):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	643 """Returns a map of the base contexts for each revision
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	644
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	645 The base contexts determine which lines are considered modified when we
38590 f068495a1c28 fix: add test case that shows why --whole with --base is useful Danny Hooper <hooper@google.com> parents: 38537 diff changeset	646 attempt to fix just the modified lines in a file. It also determines which
f068495a1c28 fix: add test case that shows why --whole with --base is useful Danny Hooper <hooper@google.com> parents: 38537 diff changeset	647 files we attempt to fix, so it is important to compute this even when
f068495a1c28 fix: add test case that shows why --whole with --base is useful Danny Hooper <hooper@google.com> parents: 38537 diff changeset	648 --whole is used.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	649 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	650 # The --base flag overrides the usual logic, and we give every revision
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	651 # exactly the set of baserevs that the user specified.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	652 if opts.get(b'base'):
48116 5ced12cfa41b errors: raise InputError on bad revset to revrange() iff provided by the user Martin von Zweigbergk <martinvonz@google.com> parents: 47924 diff changeset	653 baserevs = set(logcmdutil.revrange(repo, opts.get(b'base')))
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	654 if not baserevs:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	655 baserevs = {nullrev}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	656 basectxs = {repo[rev] for rev in baserevs}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	657 return {rev: basectxs for rev in revstofix}
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	658
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	659 # Proceed in topological order so that we can easily determine each
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	660 # revision's baserevs by looking at its parents and their baserevs.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	661 basectxs = collections.defaultdict(set)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	662 for rev in sorted(revstofix):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	663 ctx = repo[rev]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	664 for pctx in ctx.parents():
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	665 if pctx.rev() in basectxs:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	666 basectxs[rev].update(basectxs[pctx.rev()])
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	667 else:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	668 basectxs[rev].add(pctx)
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	669 return basectxs
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	670
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	671
45074 797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	672 def _prefetchfiles(repo, workqueue, basepaths):
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	673 toprefetch = set()
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	674
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	675 # Prefetch the files that will be fixed.
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	676 for srcrev, path, _dstrevs in workqueue:
f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	677 if srcrev == wdirrev:
45074 797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	678 continue
48178 f12a19d03d2c fix: reduce number of tool executions Danny Hooper <hooper@google.com> parents: 48116 diff changeset	679 toprefetch.add((srcrev, path))
45074 797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	680
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	681 # Prefetch the base contents for lineranges().
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	682 for (baserev, fixrev, path), basepath in basepaths.items():
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	683 toprefetch.add((baserev, basepath))
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	684
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	685 if toprefetch:
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	686 scmutil.prefetchfiles(
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	687 repo,
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	688 [
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	689 (rev, scmutil.matchfiles(repo, [path]))
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	690 for rev, path in toprefetch
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	691 ],
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	692 )
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	693
797ef6f8295e fix: prefetch file contents Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 45073 diff changeset	694
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	695 def fixfile(ui, repo, opts, fixers, fixctx, path, basepaths, basectxs):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	696 """Run any configured fixers that should affect the file in this context
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	697
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	698 Returns the file content that results from applying the fixers in some order
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	699 starting with the file's content in the fixctx. Fixers that support line
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	700 ranges will affect lines that have changed relative to any of the basectxs
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	701 (i.e. they will only avoid lines that are common to all basectxs).
38967 a009589cd32a fix: determine fixer tool failure by exit code instead of stderr Danny Hooper <hooper@google.com> parents: 38950 diff changeset	702
a009589cd32a fix: determine fixer tool failure by exit code instead of stderr Danny Hooper <hooper@google.com> parents: 38950 diff changeset	703 A fixer tool's stdout will become the file's new content if and only if it
42673 74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	704 exits with code zero. The fixer tool's working directory is the repository's
74b4cd091e0d fix: run fixer tools in the repo root as cwd so they can use the working copy Danny Hooper <hooper@google.com> parents: 42655 diff changeset	705 root.
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	706 """
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	707 metadata = {}
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	708 newdata = fixctx[path].data()
43105 649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/ Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	709 for fixername, fixer in pycompat.iteritems(fixers):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	710 if fixer.affects(opts, fixctx, path):
45073 54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	711 ranges = lineranges(
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	712 opts, path, basepaths, basectxs, fixctx, newdata
54009f8c3e25 fix: obtain base paths before starting workers Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 44856 diff changeset	713 )
42758 e9f503074044 fix: pass line ranges as value instead of callback Danny Hooper <hooper@google.com> parents: 42757 diff changeset	714 command = fixer.command(ui, path, ranges)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	715 if command is None:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	716 continue
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	717 ui.debug(b'subprocess: %s\n' % (command,))
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	718 proc = subprocess.Popen(
39836 f1d6021453c2 py3: remove a couple of superfluous calls to pycompat.rapply() Matt Harbison <matt_harbison@yahoo.com> parents: 39826 diff changeset	719 procutil.tonativestr(command),
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	720 shell=True,
43193 2d1f9880af1b py3: convert cwd to native string when running `fix` Matt Harbison <matt_harbison@yahoo.com> parents: 43117 diff changeset	721 cwd=procutil.tonativestr(repo.root),
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	722 stdin=subprocess.PIPE,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	723 stdout=subprocess.PIPE,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	724 stderr=subprocess.PIPE,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	725 )
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	726 stdout, stderr = proc.communicate(newdata)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	727 if stderr:
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	728 showstderr(ui, fixctx.rev(), fixername, stderr)
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	729 newerdata = stdout
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	730 if fixer.shouldoutputmetadata():
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	731 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	732 metadatajson, newerdata = stdout.split(b'\0', 1)
43380 579672b347d2 py3: define and use json.loads polyfill Gregory Szorc <gregory.szorc@gmail.com> parents: 43227 diff changeset	733 metadata[fixername] = pycompat.json_loads(metadatajson)
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	734 except ValueError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	735 ui.warn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	736 _(b'ignored invalid output from fixer tool: %s\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	737 % (fixername,)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	738 )
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	739 continue
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	740 else:
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	741 metadata[fixername] = None
38967 a009589cd32a fix: determine fixer tool failure by exit code instead of stderr Danny Hooper <hooper@google.com> parents: 38950 diff changeset	742 if proc.returncode == 0:
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	743 newdata = newerdata
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	744 else:
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	745 if not stderr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	746 message = _(b'exited with status %d\n') % (proc.returncode,)
40532 93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	747 showstderr(ui, fixctx.rev(), fixername, message)
93bab80993f4 fix: add a config to abort when a fixer tool fails Danny Hooper <hooper@google.com> parents: 40431 diff changeset	748 checktoolfailureaction(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	749 ui,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	750 _(b'no fixes will be applied'),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	751 hint=_(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	752 b'use --config fix.failure=continue to apply any '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	753 b'successful fixes anyway'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	754 ),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	755 )
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	756 return metadata, newdata
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	757
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	758
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	759 def showstderr(ui, rev, fixername, stderr):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	760 """Writes the lines of the stderr string as warnings on the ui
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	761
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	762 Uses the revision number and fixername to give more context to each line of
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	763 the error message. Doesn't include file names, since those take up a lot of
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	764 space and would tend to be included in the error message if they were
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	765 relevant.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	766 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	767 for line in re.split(b'[\r\n]+', stderr):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	768 if line:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	769 ui.warn(b'[')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	770 if rev is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	771 ui.warn(_(b'wdir'), label=b'evolve.rev')
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	772 else:
43501 b2f95f9d3588 fix: replace str() by b'%d' for formatting integer Martin von Zweigbergk <martinvonz@google.com> parents: 43380 diff changeset	773 ui.warn(b'%d' % rev, label=b'evolve.rev')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	774 ui.warn(b'] %s: %s\n' % (fixername, line))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	775
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	776
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	777 def writeworkingdir(repo, ctx, filedata, replacements):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	778 """Write new content to the working copy and check out the new p1 if any
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	779
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	780 We check out a new revision if and only if we fixed something in both the
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	781 working directory and its parent revision. This avoids the need for a full
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	782 update/merge, and means that the working directory simply isn't affected
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	783 unless the --working-dir flag is given.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	784
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	785 Directly updates the dirstate for the affected files.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	786 """
43105 649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/ Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	787 for path, data in pycompat.iteritems(filedata):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	788 fctx = ctx[path]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	789 fctx.write(data, fctx.flags())
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	790
47766 3feda1e779d4 fix: rewrite writeworkingdir() to explicitly not work with merges Martin von Zweigbergk <martinvonz@google.com> parents: 47735 diff changeset	791 oldp1 = repo.dirstate.p1()
3feda1e779d4 fix: rewrite writeworkingdir() to explicitly not work with merges Martin von Zweigbergk <martinvonz@google.com> parents: 47735 diff changeset	792 newp1 = replacements.get(oldp1, oldp1)
3feda1e779d4 fix: rewrite writeworkingdir() to explicitly not work with merges Martin von Zweigbergk <martinvonz@google.com> parents: 47735 diff changeset	793 if newp1 != oldp1:
47886 86a60679cf61 fix: again allow formatting the working copy while merging Martin von Zweigbergk <martinvonz@google.com> parents: 47797 diff changeset	794 assert repo.dirstate.p2() == nullid
47767 66ad7e32011f fix: use scmutil.movedirstate() instead of reimplementing Martin von Zweigbergk <martinvonz@google.com> parents: 47766 diff changeset	795 with repo.dirstate.parentchange():
66ad7e32011f fix: use scmutil.movedirstate() instead of reimplementing Martin von Zweigbergk <martinvonz@google.com> parents: 47766 diff changeset	796 scmutil.movedirstate(repo, repo[newp1])
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	797
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	798
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	799 def replacerev(ui, repo, ctx, filedata, replacements):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	800 """Commit a new revision like the given one, but with file content changes
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	801
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	802 "ctx" is the original revision to be replaced by a modified one.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	803
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	804 "filedata" is a dict that maps paths to their new file content. All other
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	805 paths will be recreated from the original revision without changes.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	806 "filedata" may contain paths that didn't exist in the original revision;
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	807 they will be added.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	808
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	809 "replacements" is a dict that maps a single node to a single node, and it is
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	810 updated to indicate the original revision is replaced by the newly created
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	811 one. No entry is added if the replacement's node already exists.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	812
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	813 The new revision has the same parents as the old one, unless those parents
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	814 have already been replaced, in which case those replacements are the parents
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	815 of this new revision. Thus, if revisions are replaced in topological order,
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	816 there is no need to rebase them into the original topology later.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	817 """
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	818
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	819 p1rev, p2rev = repo.changelog.parentrevs(ctx.rev())
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	820 p1ctx, p2ctx = repo[p1rev], repo[p2rev]
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	821 newp1node = replacements.get(p1ctx.node(), p1ctx.node())
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	822 newp2node = replacements.get(p2ctx.node(), p2ctx.node())
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	823
40570 ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	824 # We don't want to create a revision that has no changes from the original,
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	825 # but we should if the original revision's parent has been replaced.
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	826 # Otherwise, we would produce an orphan that needs no actual human
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	827 # intervention to evolve. We can't rely on commit() to avoid creating the
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	828 # un-needed revision because the extra field added below produces a new hash
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	829 # regardless of file content changes.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	830 if (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	831 not filedata
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	832 and p1ctx.node() not in replacements
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	833 and p2ctx.node() not in replacements
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	834 ):
40570 ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	835 return
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	836
ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	837 extra = ctx.extra().copy()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	838 extra[b'fix_source'] = ctx.hex()
40570 ad71c792a8d8 fix: add extra field to fixed revisions to avoid creating obsolescence cycles Danny Hooper <hooper@google.com> parents: 40566 diff changeset	839
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	840 wctx = context.overlayworkingctx(repo)
44092 833210fbd900 graftcopies: remove `skip` and `repo` arguments Martin von Zweigbergk <martinvonz@google.com> parents: 44090 diff changeset	841 wctx.setbase(repo[newp1node])
44271 c791ed6a2154 merge: introduce a revert_to() for that use-case Martin von Zweigbergk <martinvonz@google.com> parents: 44129 diff changeset	842 merge.revert_to(ctx, wc=wctx)
44092 833210fbd900 graftcopies: remove `skip` and `repo` arguments Martin von Zweigbergk <martinvonz@google.com> parents: 44090 diff changeset	843 copies.graftcopies(wctx, ctx, ctx.p1())
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	844
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	845 for path in filedata.keys():
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	846 fctx = ctx[path]
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	847 copysource = fctx.copysource()
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	848 wctx.write(path, filedata[path], flags=fctx.flags())
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	849 if copysource:
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	850 wctx.markcopied(path, copysource)
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	851
45713 04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	852 desc = rewriteutil.update_hash_refs(
04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	853 repo,
04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	854 ctx.description(),
04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	855 {oldnode: [newnode] for oldnode, newnode in replacements.items()},
04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	856 )
04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	857
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	858 memctx = wctx.tomemctx(
45713 04de8a1ec08f fix: update commit hash references in the new commits Matt Harbison <matt_harbison@yahoo.com> parents: 45218 diff changeset	859 text=desc,
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	860 branch=ctx.branch(),
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	861 extra=extra,
38423 32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	862 date=ctx.date(),
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	863 parents=(newp1node, newp2node),
eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	864 user=ctx.user(),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	865 )
43962 eebdd6709868 fix: fix handling of merge commits by using overlayworkingctx Kyle Lippincott <spectral@google.com> parents: 43936 diff changeset	866
38423 32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	867 sucnode = memctx.commit()
32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	868 prenode = ctx.node()
32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	869 if prenode == sucnode:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	870 ui.debug(b'node %s already existed\n' % (ctx.hex()))
38423 32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	871 else:
32fba6fe893d scmutil: make cleanupnodes optionally also fix the phase Martin von Zweigbergk <martinvonz@google.com> parents: 38420 diff changeset	872 replacements[ctx.node()] = sucnode
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	873
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	874
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	875 def getfixers(ui):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	876 """Returns a map of configured fixer tools indexed by their names
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	877
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	878 Each value is a Fixer object with methods that implement the behavior of the
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	879 fixer's config suboptions. Does not validate the config values.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	880 """
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	881 fixers = {}
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	882 for name in fixernames(ui):
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	883 enabled = ui.configbool(b'fix', name + b':enabled')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	884 command = ui.config(b'fix', name + b':command')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	885 pattern = ui.config(b'fix', name + b':pattern')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	886 linerange = ui.config(b'fix', name + b':linerange')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	887 priority = ui.configint(b'fix', name + b':priority')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	888 metadata = ui.configbool(b'fix', name + b':metadata')
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	889 skipclean = ui.configbool(b'fix', name + b':skipclean')
42655 2987d015aba4 fix: ignore fixer tool configurations that are missing patterns Danny Hooper <hooper@google.com> parents: 42653 diff changeset	890 # Don't use a fixer if it has no pattern configured. It would be
2987d015aba4 fix: ignore fixer tool configurations that are missing patterns Danny Hooper <hooper@google.com> parents: 42653 diff changeset	891 # dangerous to let it affect all files. It would be pointless to let it
2987d015aba4 fix: ignore fixer tool configurations that are missing patterns Danny Hooper <hooper@google.com> parents: 42653 diff changeset	892 # affect no files. There is no reasonable subset of files to use as the
2987d015aba4 fix: ignore fixer tool configurations that are missing patterns Danny Hooper <hooper@google.com> parents: 42653 diff changeset	893 # default.
43220 d3d1a3afe7aa fix: warn when a fixer doesn't have a configured command Martin von Zweigbergk <martinvonz@google.com> parents: 43219 diff changeset	894 if command is None:
d3d1a3afe7aa fix: warn when a fixer doesn't have a configured command Martin von Zweigbergk <martinvonz@google.com> parents: 43219 diff changeset	895 ui.warn(
d3d1a3afe7aa fix: warn when a fixer doesn't have a configured command Martin von Zweigbergk <martinvonz@google.com> parents: 43219 diff changeset	896 _(b'fixer tool has no command configuration: %s\n') % (name,)
d3d1a3afe7aa fix: warn when a fixer doesn't have a configured command Martin von Zweigbergk <martinvonz@google.com> parents: 43219 diff changeset	897 )
d3d1a3afe7aa fix: warn when a fixer doesn't have a configured command Martin von Zweigbergk <martinvonz@google.com> parents: 43219 diff changeset	898 elif pattern is None:
42655 2987d015aba4 fix: ignore fixer tool configurations that are missing patterns Danny Hooper <hooper@google.com> parents: 42653 diff changeset	899 ui.warn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	900 _(b'fixer tool has no pattern configuration: %s\n') % (name,)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	901 )
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	902 elif not enabled:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	903 ui.debug(b'ignoring disabled fixer tool: %s\n' % (name,))
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	904 else:
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	905 fixers[name] = Fixer(
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	906 command, pattern, linerange, priority, metadata, skipclean
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	907 )
40566 b9557567cc3f fix: add suboption for configuring execution order of tools Danny Hooper <hooper@google.com> parents: 40533 diff changeset	908 return collections.OrderedDict(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	909 sorted(fixers.items(), key=lambda item: item[1]._priority, reverse=True)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	910 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	911
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	912
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	913 def fixernames(ui):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	914 """Returns the names of [fix] config options that have suboptions"""
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	915 names = set()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	916 for k, v in ui.configitems(b'fix'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	917 if b':' in k:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	918 names.add(k.split(b':', 1)[0])
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	919 return names
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	920
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	921
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	922 class Fixer(object):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	923 """Wraps the raw config values for a fixer with methods"""
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	924
43219 0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	925 def __init__(
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	926 self, command, pattern, linerange, priority, metadata, skipclean
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	927 ):
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	928 self._command = command
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	929 self._pattern = pattern
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	930 self._linerange = linerange
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	931 self._priority = priority
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	932 self._metadata = metadata
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	933 self._skipclean = skipclean
0101db49606f fix: make Fixer initialization more explicit for clarity Martin von Zweigbergk <martinvonz@google.com> parents: 43218 diff changeset	934
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	935 def affects(self, opts, fixctx, path):
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	936 """Should this fixer run on the file at the given path and context?"""
43227 f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	937 repo = fixctx.repo()
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	938 matcher = matchmod.match(
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	939 repo.root, repo.root, [self._pattern], ctx=fixctx
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	940 )
f02d3c0eed18 fix: match patterns relative to root Martin von Zweigbergk <martinvonz@google.com> parents: 43221 diff changeset	941 return matcher(path)
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	942
42194 0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	943 def shouldoutputmetadata(self):
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	944 """Should the stdout of this fixer start with JSON and a null byte?"""
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	945 return self._metadata
0da689a60163 fix: allow fixer tools to return metadata in addition to the file content Danny Hooper <hooper@google.com> parents: 42009 diff changeset	946
42758 e9f503074044 fix: pass line ranges as value instead of callback Danny Hooper <hooper@google.com> parents: 42757 diff changeset	947 def command(self, ui, path, ranges):
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	948 """A shell command to use to invoke this fixer on the given file/lines
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	949
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	950 May return None if there is no appropriate command to run for the given
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	951 parameters.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	952 """
37774 d6970628b95f fix: use templater to substitute values in command string Yuya Nishihara <yuya@tcha.org> parents: 37618 diff changeset	953 expand = cmdutil.rendercommandtemplate
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	954 parts = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	955 expand(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	956 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	957 self._command,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	958 {b'rootpath': path, b'basename': os.path.basename(path)},
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	959 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	960 ]
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	961 if self._linerange:
42756 ed0da6e0d6ee fix: allow tools to use :linerange, but also run if a file is unchanged Danny Hooper <hooper@google.com> parents: 42673 diff changeset	962 if self._skipclean and not ranges:
37183 ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	963 # No line ranges to fix, so don't run the fixer.
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	964 return None
ded5ea279a93 fix: new extension for automatically modifying file contents Danny Hooper <hooper@google.com> parents: diff changeset	965 for first, last in ranges:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	966 parts.append(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	967 expand(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	968 ui, self._linerange, {b'first': first, b'last': last}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	969 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43058 diff changeset	970 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	971 return b' '.join(parts)

Mercurial > public > mercurial-scm > hg

annotate hgext/fix.py @ 48178:f12a19d03d2c