Importing ssl After Reinitializing Crashes

[ericsnowcurrently]

Crash report

What happened?

Using 3.13b4 and main (5592399):

$ Programs/_testembed test_repeated_init_exec 'import ssl'
--- Loop #1 ---
--- Loop #2 ---
Segmentation fault (core dumped)

Using 3.12:

$ Programs/_testembed test_repeated_init_exec 'import ssl'
--- Loop #1 ---
--- Loop #2 ---
_testembed: Python/getargs.c:2052: parser_init: Assertion `parser->kwtuple != NULL' failed.
Aborted (core dumped)

The 3.12 output implies a problem related to argument clinic and the kwarg names tuple that gets cached.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-122334: Fix crash when importing ssl after re-initialization #122481
• [3.12] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) #122495
• [3.13] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) #122614
• gh-122334: Fix test_embed failure when missing _ssl module #122630
• [3.13] gh-122334: Fix test_embed failure when missing _ssl module (GH-122630) #122647
• [3.12] gh-122334: Fix test_embed failure when missing _ssl module (GH-122630) #122648

[neonene]

The following command crashes as well:

_testembed test_repeated_init_exec "import _ssl; _ssl.txt2obj(txt='1.3')"

Related clinic code (actually unchanged since Python 3.5):

static PyObject *
_ssl_txt2obj(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
    ...
    static const char * const _keywords[] = {"txt", "name", NULL};
    static _PyArg_Parser _parser = {
        .keywords = _keywords,
        .fname = "txt2obj",
        .kwtuple = KWTUPLE,  // KWTUPLE: NULL
    };
    ...
    args = _PyArg_UnpackKeywords(..., &_parser, ...);  // segfault

Python/getargs.c:

PyObject * const *
_PyArg_UnpackKeywords(..., struct _PyArg_Parser *parser, ...)
{
    PyObject *kwtuple;
    ...
    if (parser_init(parser) < 0) {  // set `parser->kwtuple` only once
        return NULL;
    }
    kwtuple = parser->kwtuple;      // get NULL after restarting interp
    ...
    maxargs = posonly + (int)PyTuple_GET_SIZE(kwtuple);  // segfault

Currently, _PyArg_Fini() clears parser->kwtuple only:

void
_PyArg_Fini(void)
{
    struct _PyArg_Parser *tmp, *s = _PyRuntime.getargs.static_parsers;
    while (s) {
        tmp = s->next;
        s->next = NULL;
        parser_clear(s);
        s = tmp;
    }
    _PyRuntime.getargs.static_parsers = NULL;
}

// 3.13+
static void
parser_clear(struct _PyArg_Parser *parser)
{
    if (parser->is_kwtuple_owned) {
        Py_CLEAR(parser->kwtuple);
    }
}

// 3.12
static void
parser_clear(struct _PyArg_Parser *parser)
{
    if (parser->initialized == 1) {
        Py_CLEAR(parser->kwtuple);
    }
}

IIUC, clearing parser->once.v or parser->initialized there will allow parser_init() to run again, which would work around the issue.

[ericsnowcurrently]

Good sleuthing!

[neonene]

Some buildbots fail the test_embed without building _ssl module.
https://buildbot.python.org/#/builders/1366/builds/1792