Mercurial > public > mercurial-scm > hg-stable
annotate tests/test-stdio.py @ 45095:8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Python 3 offers different kind of streams and it?s not guaranteed for all of
them that calling write() writes all bytes.
When Python is started in unbuffered mode, sys.std{out,err}.buffer are
instances of io.FileIO, whose write() can write less bytes for
platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
write less if interrupted by a signal; when writing to Windows consoles, it?s
limited to 32767 bytes to avoid the "not enough space" error). This can lead to
silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
not be a buffered stream) and when using the text streams sys.std{out,err}
(I?ve created a CPython bug report for that:
https://bugs.python.org/issue41221).
Python may fix the problem at some point. For now, we implement our own wrapper
for procutil.std{out,err} that calls the raw stream?s write() method until all
bytes have been written. We don?t use sys.std{out,err} for larger writes, so I
think it?s not worth the effort to patch them.
author | Manuel Jacob <me@manueljacob.de> |
---|---|
date | Fri, 10 Jul 2020 12:27:58 +0200 |
parents | a59aab6078eb |
children | e9e452eafbfb |
rev | line source |
---|---|
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
1 #!/usr/bin/env python |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
2 """ |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
3 Tests the buffering behavior of stdio streams in `mercurial.utils.procutil`. |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
4 """ |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
5 from __future__ import absolute_import |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
6 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
7 import contextlib |
45069
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
8 import errno |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
9 import os |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
10 import signal |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
11 import subprocess |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
12 import sys |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
13 import unittest |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
14 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
15 from mercurial import pycompat |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
16 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
17 |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
18 TEST_BUFFERING_CHILD_SCRIPT = r''' |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
19 import os |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
20 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
21 from mercurial import dispatch |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
22 from mercurial.utils import procutil |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
23 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
24 dispatch.initstdio() |
45047
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
25 procutil.{stream}.write(b'aaa') |
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
26 os.write(procutil.{stream}.fileno(), b'[written aaa]') |
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
27 procutil.{stream}.write(b'bbb\n') |
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
28 os.write(procutil.{stream}.fileno(), b'[written bbb\\n]') |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
29 ''' |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
30 UNBUFFERED = b'aaa[written aaa]bbb\n[written bbb\\n]' |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
31 LINE_BUFFERED = b'[written aaa]aaabbb\n[written bbb\\n]' |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
32 FULLY_BUFFERED = b'[written aaa][written bbb\\n]aaabbb\n' |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
33 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
34 |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
35 TEST_LARGE_WRITE_CHILD_SCRIPT = r''' |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
36 import signal |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
37 import sys |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
38 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
39 from mercurial import dispatch |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
40 from mercurial.utils import procutil |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
41 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
42 signal.signal(signal.SIGINT, lambda *x: None) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
43 dispatch.initstdio() |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
44 procutil.{stream}.write(b'x' * 1048576) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
45 ''' |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
46 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
47 |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
48 @contextlib.contextmanager |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
49 def _closing(fds): |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
50 try: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
51 yield |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
52 finally: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
53 for fd in fds: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
54 try: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
55 os.close(fd) |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
56 except EnvironmentError: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
57 pass |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
58 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
59 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
60 @contextlib.contextmanager |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
61 def _pipes(): |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
62 rwpair = os.pipe() |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
63 with _closing(rwpair): |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
64 yield rwpair |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
65 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
66 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
67 @contextlib.contextmanager |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
68 def _ptys(): |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
69 if pycompat.iswindows: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
70 raise unittest.SkipTest("PTYs are not supported on Windows") |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
71 import pty |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
72 import tty |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
73 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
74 rwpair = pty.openpty() |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
75 with _closing(rwpair): |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
76 tty.setraw(rwpair[0]) |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
77 yield rwpair |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
78 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
79 |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
80 def _readall(fd, buffer_size, initial_buf=None): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
81 buf = initial_buf or [] |
45069
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
82 while True: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
83 try: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
84 s = os.read(fd, buffer_size) |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
85 except OSError as e: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
86 if e.errno == errno.EIO: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
87 # If the child-facing PTY got closed, reading from the |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
88 # parent-facing PTY raises EIO. |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
89 break |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
90 raise |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
91 if not s: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
92 break |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
93 buf.append(s) |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
94 return b''.join(buf) |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
95 |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
96 |
45047
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
97 class TestStdio(unittest.TestCase): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
98 def _test( |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
99 self, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
100 child_script, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
101 stream, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
102 rwpair_generator, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
103 check_output, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
104 python_args=[], |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
105 ): |
45047
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
106 assert stream in ('stdout', 'stderr') |
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
107 with rwpair_generator() as (stream_receiver, child_stream), open( |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
108 os.devnull, 'rb' |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
109 ) as child_stdin: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
110 proc = subprocess.Popen( |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
111 [sys.executable] + python_args + ['-c', child_script], |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
112 stdin=child_stdin, |
45047
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
113 stdout=child_stream if stream == 'stdout' else None, |
359884685eab
tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45046
diff
changeset
|
114 stderr=child_stream if stream == 'stderr' else None, |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
115 ) |
45069
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
116 try: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
117 os.close(child_stream) |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
118 check_output(stream_receiver, proc) |
45070
9172fd511999
tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents:
45069
diff
changeset
|
119 except: # re-raises |
9172fd511999
tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents:
45069
diff
changeset
|
120 proc.terminate() |
9172fd511999
tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents:
45069
diff
changeset
|
121 raise |
45069
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
122 finally: |
8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents:
45048
diff
changeset
|
123 retcode = proc.wait() |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
124 self.assertEqual(retcode, 0) |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
125 |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
126 def _test_buffering( |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
127 self, stream, rwpair_generator, expected_output, python_args=[] |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
128 ): |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
129 def check_output(stream_receiver, proc): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
130 self.assertEqual(_readall(stream_receiver, 1024), expected_output) |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
131 |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
132 self._test( |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
133 TEST_BUFFERING_CHILD_SCRIPT.format(stream=stream), |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
134 stream, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
135 rwpair_generator, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
136 check_output, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
137 python_args, |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
138 ) |
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
139 |
45071
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
140 def test_buffering_stdout_pipes(self): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
141 self._test_buffering('stdout', _pipes, FULLY_BUFFERED) |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
142 |
45071
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
143 def test_buffering_stdout_ptys(self): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
144 self._test_buffering('stdout', _ptys, LINE_BUFFERED) |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
145 |
45071
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
146 def test_buffering_stdout_pipes_unbuffered(self): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
147 self._test_buffering('stdout', _pipes, UNBUFFERED, python_args=['-u']) |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
148 |
45071
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
149 def test_buffering_stdout_ptys_unbuffered(self): |
45079
a59aab6078eb
tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents:
45078
diff
changeset
|
150 self._test_buffering('stdout', _ptys, UNBUFFERED, python_args=['-u']) |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
151 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
152 if not pycompat.ispy3 and not pycompat.iswindows: |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
153 # On Python 2 on non-Windows, we manually open stdout in line-buffered |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
154 # mode if connected to a TTY. We should check if Python was configured |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
155 # to use unbuffered stdout, but it's hard to do that. |
45071
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
156 test_buffering_stdout_ptys_unbuffered = unittest.expectedFailure( |
bc05c13e246f
tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents:
45070
diff
changeset
|
157 test_buffering_stdout_ptys_unbuffered |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
158 ) |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
159 |
45095
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
160 def _test_large_write(self, stream, rwpair_generator, python_args=[]): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
161 if not pycompat.ispy3 and pycompat.isdarwin: |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
162 # Python 2 doesn't always retry on EINTR, but the libc might retry. |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
163 # So far, it was observed only on macOS that EINTR is raised at the |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
164 # Python level. As Python 2 support will be dropped soon-ish, we |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
165 # won't attempt to fix it. |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
166 raise unittest.SkipTest("raises EINTR on macOS") |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
167 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
168 def check_output(stream_receiver, proc): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
169 if not pycompat.iswindows: |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
170 # On Unix, we can provoke a partial write() by interrupting it |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
171 # by a signal handler as soon as a bit of data was written. |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
172 # We test that write() is called until all data is written. |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
173 buf = [os.read(stream_receiver, 1)] |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
174 proc.send_signal(signal.SIGINT) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
175 else: |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
176 # On Windows, there doesn't seem to be a way to cause partial |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
177 # writes. |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
178 buf = [] |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
179 self.assertEqual( |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
180 _readall(stream_receiver, 131072, buf), b'x' * 1048576 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
181 ) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
182 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
183 self._test( |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
184 TEST_LARGE_WRITE_CHILD_SCRIPT.format(stream=stream), |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
185 stream, |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
186 rwpair_generator, |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
187 check_output, |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
188 python_args, |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
189 ) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
190 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
191 def test_large_write_stdout_pipes(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
192 self._test_large_write('stdout', _pipes) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
193 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
194 def test_large_write_stdout_ptys(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
195 self._test_large_write('stdout', _ptys) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
196 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
197 def test_large_write_stdout_pipes_unbuffered(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
198 self._test_large_write('stdout', _pipes, python_args=['-u']) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
199 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
200 def test_large_write_stdout_ptys_unbuffered(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
201 self._test_large_write('stdout', _ptys, python_args=['-u']) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
202 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
203 def test_large_write_stderr_pipes(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
204 self._test_large_write('stderr', _pipes) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
205 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
206 def test_large_write_stderr_ptys(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
207 self._test_large_write('stderr', _ptys) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
208 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
209 def test_large_write_stderr_pipes_unbuffered(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
210 self._test_large_write('stderr', _pipes, python_args=['-u']) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
211 |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
212 def test_large_write_stderr_ptys_unbuffered(self): |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
213 self._test_large_write('stderr', _ptys, python_args=['-u']) |
8e04607023e5
procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents:
45079
diff
changeset
|
214 |
45041
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
215 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
216 if __name__ == '__main__': |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
217 import silenttestrunner |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
218 |
c7d109c400a4
tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff
changeset
|
219 silenttestrunner.main(__name__) |