[macOS] ``test_builtin.PtyTests.test_input_tty`` hangs if ``rlcompleter`` is imported

[Eclips4]

Bug report

Bug description:

./python.exe -m test
...many lines
0:05:14 load avg: 4.04 [ 87/472] test_bool
0:05:14 load avg: 4.04 [ 88/472] test_buffer
0:05:15 load avg: 3.80 [ 89/472] test_bufio
0:05:16 load avg: 3.80 [ 90/472] test_builtin
/Users/admin/Projects/cpython/Lib/pty.py:95: DeprecationWarning: This process (pid=47600) is multi-threaded, use of forkpty() may lead to deadlocks in the child.
  pid, fd = os.forkpty()

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-116402: Avoid readline in test_builtin TTY input tests #122447
• [3.13] gh-116402: Avoid readline in test_builtin TTY input tests (GH-122447) #122472

[Eclips4]

Reduced amount of tests to trigger the hang:

 ./python.exe -m test test___all__ test_builtin 
Using random seed: 2282378935
0:00:00 load avg: 1.58 Run 2 tests sequentially
0:00:00 load avg: 1.58 [1/2] test___all__
0:00:00 load avg: 1.58 [2/2] test_builtin
^C

== Tests result: INTERRUPTED ==

1 test omitted:
    test_builtin

1 test OK.

Test suite interrupted by signal SIGINT.

Total duration: 18.0 sec
Total tests: run=1
Total test files: run=1/2
Result: INTERRUPTED

[sobolevn]

Minimal number of tests:

test.test___all__.AllTest.test_all
test.test_builtin.PtyTests.test_input_tty

It gets stuck here:

cpython/Lib/test/test_builtin.py

Line 2282 in d2f1b0e

line = rpipe.readline().strip()

[sobolevn]

Since test_all actually imports all modules and executes them, the problem can be in any module :(

[sobolevn]

I think that it would be easier to bisect when running only two test cases at least!

[Eclips4]

If we skip import of rlcompleter in the test_all, then everything will be fine:

diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py
index 19dcbb207e..c5bf4bb1ff 100644
--- a/Lib/test/test___all__.py
+++ b/Lib/test/test___all__.py
@@ -111,6 +111,8 @@ def test_all(self):
         lib_dir = os.path.dirname(os.path.dirname(__file__))
         for path, modname in self.walk_modules(lib_dir, ""):
             m = modname
+            if m == "rlcompleter":
+                continue
             denied = False
             while m:
                 if m in denylist:

./python.exe -m test test___all__ test_builtin
Using random seed: 1172128194
0:00:00 load avg: 4.37 Run 2 tests sequentially
0:00:00 load avg: 4.37 [1/2] test___all__
0:00:01 load avg: 4.37 [2/2] test_builtin

== Tests result: SUCCESS ==

All 2 tests OK.

Total duration: 1.7 sec
Total tests: run=133 skipped=1
Total test files: run=2/2
Result: SUCCESS

[Eclips4]

Seems like problem in this import:
(sources of Lib/rlcompleter.py)

try:
    import readline
except ImportError:
    _readline_available = False
else:
    readline.set_completer(Completer().complete)
    # Release references early at shutdown (the readline module's
    # contents are quasi-immortal, and the completer function holds a
    # reference to globals).
    atexit.register(lambda: readline.set_completer(None))
    _readline_available = True

[vstinner]

If a test fails with a completer callback is set, you can try to:

completer = readline.get_completer()
try:
    readline.set_completer(None)
   ... # run the test
finally:
    readline.set_completer(completer)

But only do it if readline is in sys.modules.

[serhiy-storchaka]

test.test_builtin.PtyTests.test_input_tty can also be affected by other tests that import rlcompleter, and importing rlcompleter can also affect other tests. There may be several old issues which will be solved when solve this issue.

I would add a code that restores (or just resets) the completer in Lib/test/libregrtest/save_env.py.

[vstinner]

readline.set_completer(None)

If you fully want to control the TTY and not be affected by previous tests, another approach is to spawn a new process.

[ronaldoussoren]

This might be a duplicate of #89050

[JelleZijlstra]

This test hangs for me on a Mac even without running any other test:

% ./python.exe -m test test_builtin -m test_input_tty -v
== CPython 3.14.0a0 (heads/pep649-inspect:ad8c51c157, May 31 2024, 08:27:05) [Clang 14.0.0 (clang-1400.0.29.202)]
== macOS-13.4.1-arm64-arm-64bit-Mach-O little-endian
== Python build: debug
== cwd: /Users/jelle/py/cpython/build/test_python_worker_85845æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1737450533
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 1.34 Run 1 test sequentially
0:00:00 load avg: 1.34 [1/1] test_builtin
test_input_tty (test.test_builtin.PtyTests.test_input_tty) ... ^C

== Tests result: INTERRUPTED ==