Error in AST recursion depth tracking change of gh-95185 #106905
Description
Bug report
The change in 0047447 seems to miss the recusrion depth adjustment in case of an error. As an example for some of the generated code:
PyObject*
ast2obj_mod(struct ast_state *state, void* _o)
{
mod_ty o = (mod_ty)_o;
PyObject *result = NULL, *value = NULL;
PyTypeObject *tp;
if (!o) {
Py_RETURN_NONE;
}
if (++state->recursion_depth > state->recursion_limit) {
PyErr_SetString(PyExc_RecursionError,
"maximum recursion depth exceeded during ast construction");
return 0;
}
switch (o->kind) {
case Module_kind:
tp = (PyTypeObject *)state->Module_type;
result = PyType_GenericNew(tp, NULL, NULL);
if (!result) goto failed;
value = ast2obj_list(state, (asdl_seq*)o->v.Module.body, ast2obj_stmt);
if (!value) goto failed;
if (PyObject_SetAttr(result, state->body, value) == -1)
goto failed;
...
}
state->recursion_depth--;
return result;
failed:
Py_XDECREF(value);
Py_XDECREF(result);
return NULL;
}
Note that the failed code path is missing the state->recursion_depth--; statement.
I found this as I'm trying to track down where spurious SystemError: AST constructor recursion depth mismatch errors in Python 3.11 are coming from. E.g.
File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 358, in unpack_ndarray_from_bytes
return unpack_ndarray(source)
^^^^^^^^^^^^^^^^^
File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 305, in unpack_ndarray
sink = PlainNumpySink(source.metadata)
^^^^^^^^^^^^^^^^^
File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 136, in __init__
dtype_ = ast.literal_eval(metadata['dtype'])
^^^^^^^^^^^^^^^^^
File "/env/lib/python3.11/ast.py", line 64, in literal_eval
node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')
^^^^^^^^^^^^^^^^^
File "/env/lib/python3.11/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
^^^^^^^^^^^^^^^^^
SystemError: AST constructor recursion depth mismatch (before=48, after=47)
Your environment
Reproduced in Python 3.11 but the code in main looks the same.
Linked PRs
- gh-106905: avoid incorrect SystemError about recursion depth mismatch #106906
- [3.12] gh-106905: avoid incorrect SystemError about recursion depth mismatch (GH-106906) #112032
- [3.11] gh-106905: Avoid incorrect SystemError about recursion depth mismatch (#106906) #112849
- gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. #113035
- [3.12] gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) #113472
- [3.11] gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) (GH-113472) #113476