test_ctypes.test_generated_structs.GeneratedTest.test_generated_data 'ManyTypesU' fails on Python 3.14 built with gcc 15

[befeleme]

Bug report

Bug description:

When running test suite on Python 3.14.0a4 built with gcc 15.0.3 on Fedora Linux 42 (Rawhide), test_generated_data fails for many of its types.
I suspect this is due to the new gcc version. This doesn't happen when Python is built with gcc 14.

Check that a ctypes struct/union matches its C equivalent. ... 
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i8', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i8', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i8', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u8', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u8', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u8', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i16', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i16', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i16', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u16', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u16', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u16', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i32', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i32', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i32', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u32', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u32', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u32', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL

Example output:

FAIL: test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='i8', value=-1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/python3.14-3.14.0_a4-build/Python-3.14.0a4/Lib/test/test_ctypes/test_generated_structs.py", line 473, in test_generated_data
    self.assertEqual(py_mem.hex(), c_mem.hex(), m)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'ff00000000000000' != 'ff8f8783ffff0000'
- ff00000000000000
+ ff8f8783ffff0000
 : <FieldInfo for ManyTypesU.i8: <Field type=c_int8, ofs=0, size=1>>
in:
union {
    int8_t i8;
    uint8_t u8;
    int16_t i16;
    uint16_t u16;
    int32_t i32;
    uint32_t u32;
    int64_t i64;
    uint64_t u64;
}

Ran 1 test in 0.178s
FAILED (failures=18, skipped=11)

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-128889: Zero out memory ctypes for generated struct layout tests #128944

[vstinner]

cc @encukou

[davidmalcolm]

Note to self: test code in question is: https://github.com/python/cpython/blob/main/Lib/test/test_ctypes/test_generated_structs.py

[hroncok]

I was gonna report that the failures do not occur with Python 3.13 -- only to realize the tests do not exist there either.

[davidmalcolm]

The failures in test_ctypes occurred for me on all archs I tried on (aarch64, ppc64le, s390x, x86_64) with gcc-15.0.0-0.2.fc42, from upstream gcc source 2024-12-03 revision d2b3fd44a4b20c573bb9c8ef5b17bdca70ea748b).

Build logs are currently here:
https://copr.fedorainfracloud.org/coprs/dmalcolm/gcc-15-smoketest-3.failed/build/8477142/

[davidmalcolm]

The package was built with 4 different configurations: "debug", "optimized", "freethreading-debug", "freethreading".

Trying manually, test_ctypes succeeds with "optimized" and "freethreading", but fails with "debug" and "freethreading-debug".

(the package build tries "debug" first in %check, and the failure there causes a fatal error of the package build without trying the other configs)

[davidmalcolm]

Something I did locally has fixed it, FWIW I was trying to regenerate Modules/_ctypes/_ctypes_test_generated.c.h and somehow it now works for all 4 configurations.

I'm not sure what I did that "fixed" it.

[encukou]

Thanks for the report! I set some time aside for ctypes each week; I'll look into this soon.
Feel free to skip the tests in Fedora for now.

Thanks for the cc, Victor! @befeleme, you can mention me directly on anything ctypes related :)

[davidmalcolm]

Hacking in some test prints and looking at the py_mem vs c_mem, it looks like the test is being over-zealous about the result.

For example here:

  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u8', value=1, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
field=<FieldInfo for ManyTypesU.u8: <Field type=c_uint8, ofs=0, size=1>>, value=0
py_mem=b'\x00\x00\x00\x00\x00\x00\x00\x00'
 c_mem=b'\x00\xca\xb6\xff\xfc\x7f\x00\x00'

...I see that the byte[0] is the same in both.

In here, with ofs=0, size=2:

  test_generated_data (test.test_ctypes.test_generated_structs.GeneratedTest.test_generated_data) (field='u8', value=0, name='ManyTypesU')
Check that a ctypes struct/union matches its C equivalent. ... FAIL
field=<FieldInfo for ManyTypesU.i16: <Field type=c_int16, ofs=0, size=2>>, value=-1
py_mem=b'\xff\xff\x00\x00\x00\x00\x00\x00'
 c_mem=b'\xff\xff\xb6\xff\xfc\x7f\x00\x00'

I see that bytes[0] and [1] are indeed the same in both.

The docstring has this:

    This does depend on the C compiler keeping padding bits zero.
    Common compilers seem to do so.

which perhaps GCC 15 isn't doing in some cases (not sure; haven't reliably reproduced this).

[davidmalcolm]

Possibly relates to this new GCC 15 optimization: expr, c: Don't clear whole unions relating to GCC bug 116416.

If so (though I haven't proven this), a workaround might be to use -fzero-init-padding-bits=unions (which is new in GCC 15, added by that patch).