GHA "Check if generated files are up to date" job failed with: No such file or directory: ./Parser/parser.new.c

[vstinner]

The GHA "Check if generated files are up to date" job failed with: No such file or directory: './Parser/parser.new.c'.

This job runs:

make regen-deepfreeze
make -j4 regen-all
make regen-stdlib-module-names

Logs:

The Makefile was updated, you may need to re-run make.
PYTHONPATH=./Tools/peg_generator python3.11 -m pegen -q c \
	./Grammar/python.gram \
	./Grammar/Tokens \
	-o ./Parser/parser.new.c
python3.11 ./Tools/build/update_file.py ./Tools/peg_generator/pegen/grammar_parser.py \
./Tools/peg_generator/pegen/grammar_parser.py.new
gcc -c -fno-strict-overflow -Wsign-compare -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -Og -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include   -fPIC -DPy_BUILD_CORE -o Programs/python.o ./Programs/python.c
Read 186 instructions/ops, 5 supers, 1 macros, and 12 families from ./Python/bytecodes.c
Wrote 186 instructions, 5 supers, and 1 macros to ./Python/generated_cases.c.h.new
python3.11 ./Tools/build/update_file.py ./Python/generated_cases.c.h ./Python/generated_cases.c.h.new
python3.11 ./Tools/build/update_file.py ./Python/opcode_metadata.h ./Python/opcode_metadata.h.new
gcc -c -fno-strict-overflow -Wsign-compare -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -Og -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include   -fPIC -DPy_BUILD_CORE -o Python/deepfreeze/deepfreeze.o Python/deepfreeze/deepfreeze.c
gcc -c -fno-strict-overflow -Wsign-compare -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -Og -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include   -fPIC -DPy_BUILD_CORE -o Python/frozen.o Python/frozen.c
python3.11 ./Tools/build/update_file.py ./Lib/keyword.py ./Lib/keyword.py.new
# Regenerate Lib/test/levenshtein_examples.json
python3.11 ./Tools/build/generate_levenshtein_examples.py ./Lib/test/levenshtein_examples.json
/home/runner/work/cpython/cpython/Lib/test/levenshtein_examples.json already exists, skipping regeneration.
To force, add --overwrite to the invocation of this tool or delete the existing file.
python3.11 ./Tools/clinic/clinic.py --make --srcdir .
python3.11 ./Tools/build/update_file.py ./Parser/parser.c ./Parser/parser.new.c
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/./Tools/clinic/clinic.py", line 5562, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/./Tools/clinic/clinic.py", line 5542, in main
    parse_file(path, verify=not ns.force)
  File "/home/runner/work/cpython/cpython/./Tools/clinic/clinic.py", line 2253, in parse_file
    with open(filename, encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: './Parser/parser.new.c'

build: https://github.com/python/cpython/actions/runs/6423320320/job/17441640730?pr=110427

Linked PRs

• gh-110429: Fix race condition in "make regen-all" #110433
• [3.12] gh-110429: Fix race condition in "make regen-all" (GH-110433) #110438
• [3.11] gh-110429: Fix race condition in "make regen-all" (GH-110433) #110439

[vstinner]

The problem is that 2 Makefile jobs are run in parallel:

• make regen-pegen
• make clinic: command python3.11 ./Tools/clinic/clinic.py --make --srcdir .

make regen-pegen creates a temporary Parser/parser.new.c job and then removes it.

make clinic looks for all .c files and then parse them.

If Parser/parser.new.c is found by clinic, but is deleted but it's used, clinic fails with a FileNotFoundError.