Mercurial > public > mercurial-scm > hg
comparison mercurial/dispatch.py @ 44653:02fa5392bab6
dispatch: force \n for newlines on sys.std* streams (BC)
The sys.std* streams behave differently on Python 3. On Python 3,
these streams are an io.TextIOWrapper that wraps a binary buffer
stored on a .buffer attribute. These TextIOWrapper instances
normalize \n to os.linesep by default. On Windows, this means
that \n is normalized to \r\n. So functions like print() which
have an implicit end='\n' will actually emit \r\n for line endings.
While most parts of Mercurial go through the ui.write() layer to
print output, some code - notably in extensions and hooks - can use
print(). If this code was using print() or otherwise writing to
sys.std* on Windows, Mercurial would emit \r\n.
In reality, pretty much everything on Windows reacts to \n just fine.
Mercurial itself doesn't emit \r\n when going through the ui layer.
Changing the sys.std* streams to not normalize line endings sounds
like a scary change. But I think it is safe. It also makes Mercurial
on Python 3 behave similarly to Python 2, which did not perform \r\n
normalization in print() by default.
.. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3
Differential Revision: https://phab.mercurial-scm.org/D8339
| author | Gregory Szorc <gregory.szorc@gmail.com> |
|---|---|
| date | Sun, 29 Mar 2020 13:06:59 -0700 |
| parents | 9d2b2df2c2ba |
| children | 9e6b86a8f438 |
comparison
equal
deleted
inserted
replaced
| 44652:3cbbfd0bfc17 | 44653:02fa5392bab6 |
|---|---|
| 8 from __future__ import absolute_import, print_function | 8 from __future__ import absolute_import, print_function |
| 9 | 9 |
| 10 import difflib | 10 import difflib |
| 11 import errno | 11 import errno |
| 12 import getopt | 12 import getopt |
| 13 import io | |
| 13 import os | 14 import os |
| 14 import pdb | 15 import pdb |
| 15 import re | 16 import re |
| 16 import signal | 17 import signal |
| 17 import sys | 18 import sys |
| 142 | 143 |
| 143 | 144 |
| 144 if pycompat.ispy3: | 145 if pycompat.ispy3: |
| 145 | 146 |
| 146 def initstdio(): | 147 def initstdio(): |
| 147 pass | 148 # stdio streams on Python 3 are io.TextIOWrapper instances proxying another |
| 149 # buffer. These streams will normalize \n to \r\n by default. Mercurial's | |
| 150 # preferred mechanism for writing output (ui.write()) uses io.BufferedWriter | |
| 151 # instances, which write to the underlying stdio file descriptor in binary | |
| 152 # mode. ui.write() uses \n for line endings and no line ending normalization | |
| 153 # is attempted through this interface. This "just works," even if the system | |
| 154 # preferred line ending is not \n. | |
| 155 # | |
| 156 # But some parts of Mercurial (e.g. hooks) can still send data to sys.stdout | |
| 157 # and sys.stderr. They will inherit the line ending normalization settings, | |
| 158 # potentially causing e.g. \r\n to be emitted. Since emitting \n should | |
| 159 # "just work," here we change the sys.* streams to disable line ending | |
| 160 # normalization, ensuring compatibility with our ui type. | |
| 161 | |
| 162 # write_through is new in Python 3.7. | |
| 163 kwargs = { | |
| 164 "newline": "\n", | |
| 165 "line_buffering": sys.stdout.line_buffering, | |
| 166 } | |
| 167 if util.safehasattr(sys.stdout, "write_through"): | |
| 168 kwargs["write_through"] = sys.stdout.write_through | |
| 169 sys.stdout = io.TextIOWrapper( | |
| 170 sys.stdout.buffer, sys.stdout.encoding, sys.stdout.errors, **kwargs | |
| 171 ) | |
| 172 | |
| 173 kwargs = { | |
| 174 "newline": "\n", | |
| 175 "line_buffering": sys.stderr.line_buffering, | |
| 176 } | |
| 177 if util.safehasattr(sys.stderr, "write_through"): | |
| 178 kwargs["write_through"] = sys.stderr.write_through | |
| 179 sys.stderr = io.TextIOWrapper( | |
| 180 sys.stderr.buffer, sys.stderr.encoding, sys.stderr.errors, **kwargs | |
| 181 ) | |
| 182 | |
| 183 # No write_through on read-only stream. | |
| 184 sys.stdin = io.TextIOWrapper( | |
| 185 sys.stdin.buffer, | |
| 186 sys.stdin.encoding, | |
| 187 sys.stdin.errors, | |
| 188 # None is universal newlines mode. | |
| 189 newline=None, | |
| 190 line_buffering=sys.stdin.line_buffering, | |
| 191 ) | |
| 148 | 192 |
| 149 def _silencestdio(): | 193 def _silencestdio(): |
| 150 for fp in (sys.stdout, sys.stderr): | 194 for fp in (sys.stdout, sys.stderr): |
| 151 # Check if the file is okay | 195 # Check if the file is okay |
| 152 try: | 196 try: |