Unhandled `BytesWarning` when `test.support.strace_helper.strace_python` fails

[andreas-schwab]

======================================================================
FAIL: test_all (test.test___all__.AllTest.test_all)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test___all__.py", line 129, in test_all
    self.check_all(modname)
    ~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test___all__.py", line 35, in check_all
    with warnings_helper.check_warnings(
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        (f".*{modname}", DeprecationWarning),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        ("", ResourceWarning),
        ^^^^^^^^^^^^^^^^^^^^^^
        quiet=True):
        ^^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/warnings_helper.py", line 185, in _filterwarnings
    raise AssertionError("unhandled warning %s" % reraise[0])
AssertionError: unhandled warning {message : BytesWarning('str() on a bytes instance'), category : 'BytesWarning', filename : '/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py', lineno : 85, line : None}

----------------------------------------------------------------------

Linked PRs

• gh-127747: Resolve BytesWarning in test.support.strace_helper #127849

[eendebakpt]

@andreas-schwab Could you provide some more details about the error? For example: which platform are you using, how did you start the tests (from the message above I gather you are running the cpython unit tests), how did you install or compile cpython, is python itself running as expected?

[andreas-schwab]

This is the actual test failure:

test test_subprocess crashed -- Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/libregrtest/single.py", line 181, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/libregrtest/single.py", line 128, in _load_run_test
    test_mod = importlib.import_module(module_name)
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1386, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1359, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1330, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 756, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test_subprocess.py", line 1843, in <module>
    class POSIXProcessTestCase(BaseTestCase):
    ...<1628 lines>...
                    self.assertNotRegex(non_vfork_result.event_bytes, br"(?i)vfork")
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test_subprocess.py", line 3420, in POSIXProcessTestCase
    @strace_helper.requires_strace()
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 166, in requires_strace
    return unittest.skipUnless(_can_strace(), "Requires working strace")
                               ~~~~~~~~~~~^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 153, in _can_strace
    res = strace_python("import sys; sys.exit(0)", [], check=False)
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 105, in strace_python
    return _make_error("Expected strace events and exit code line",
                       stripped[-50:])
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 85, in _make_error
    event_bytes=f"error({reason},details={details}) = -1".encode('utf-8'),
                                         ^^^^^^^^^
BytesWarning: str() on a bytes instance

[picnixz]

Please, tell us how Python was configured and installed and how the tests were executed.

[andreas-schwab]

https://build.opensuse.org/package/live_build_log/openSUSE:Factory:RISCV/python314/standard/riscv64

./configure --host=riscv64-suse-linux --build=riscv64-suse-linux --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-platlibdir=lib64 --docdir=/usr/share/doc/packages/python --enable-ipv6 --enable-shared --with-ensurepip=no --with-system-ffi --with-system-expat --with-lto --with-ssl-default-suites=openssl --enable-optimizations --with-system-libmpdec --enable-loadable-sqlite-extensions
make
make test

[picnixz]

Ok, my best guess is that the strace helper is not found, so something is raised. However, when we raise that error, we use details, which is likely a bytes instance and not a string one. Why? because details is probably obtained from stderr of subprocess.run(...) used to check the presence of strace, and we likely forgot to decode the output.

I'll check this tomorrow and provide a patch if needed.

[picnixz]

The error seems to be in AutoFileTests.test_syscalls_read according to your logs. Note that the test_vfork_used_when_expected is also affected and it's actually the decorator that fails.

cc @cmaloney as the one who added this decorator.

[cmaloney]

Walking through the warning log + code looks like the strace binary exists but gave unexpected output. That leads to a call to the _make_error helper (https://github.com/python/cpython/blob/main/Lib/test/support/strace_helper.py#L103-L109) where details is a bytes rather than an OSError or str. I'd been using f-strings to format the output, and there is an implicit conversion from bytes -> str there... Not sure if there's a common / preferred way to fix, will experiment a bit.