From 9ad25085a88b2bb71d59a673d208d46c26c94f29 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 22 Jun 2021 16:58:35 +0200 Subject: [PATCH 1/3] Remove documentation for the removed PyParser_* C API --- Doc/c-api/veryhigh.rst | 36 ------------------------------------ Doc/data/refcounts.dat | 26 -------------------------- Doc/faq/extending.rst | 35 ++--------------------------------- 3 files changed, 2 insertions(+), 95 deletions(-) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 0f760eaa7ad578..3354a2b976f343 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -185,42 +185,6 @@ the same library that the Python runtime is using. :c:func:`PyMem_RawRealloc`, instead of being allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. - -.. c:function:: struct _node* PyParser_SimpleParseString(const char *str, int start) - - This is a simplified interface to - :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving *filename* set - to ``NULL`` and *flags* set to ``0``. - - -.. c:function:: struct _node* PyParser_SimpleParseStringFlags( const char *str, int start, int flags) - - This is a simplified interface to - :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving *filename* set - to ``NULL``. - - -.. c:function:: struct _node* PyParser_SimpleParseStringFlagsFilename( const char *str, const char *filename, int start, int flags) - - Parse Python source code from *str* using the start token *start* according to - the *flags* argument. The result can be used to create a code object which can - be evaluated efficiently. This is useful if a code fragment must be evaluated - many times. *filename* is decoded from the :term:`filesystem encoding and - error handler`. - - -.. c:function:: struct _node* PyParser_SimpleParseFile(FILE *fp, const char *filename, int start) - - This is a simplified interface to :c:func:`PyParser_SimpleParseFileFlags` below, - leaving *flags* set to ``0``. - - -.. c:function:: struct _node* PyParser_SimpleParseFileFlags(FILE *fp, const char *filename, int start, int flags) - - Similar to :c:func:`PyParser_SimpleParseStringFlagsFilename`, but the Python - source code is read from *fp* instead of an in-memory string. - - .. c:function:: PyObject* PyRun_String(const char *str, int start, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyRun_StringFlags` below, leaving diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index f3aacb8ed8ed0d..22dae0c1ef1afd 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -1791,32 +1791,6 @@ PyObject_TypeCheck:int::: PyObject_TypeCheck:PyObject*:o:0: PyObject_TypeCheck:PyTypeObject*:type:0: -PyParser_SimpleParseFile:struct _node*::: -PyParser_SimpleParseFile:FILE*:fp:: -PyParser_SimpleParseFile:const char*:filename:: -PyParser_SimpleParseFile:int:start:: - -PyParser_SimpleParseFileFlags:struct _node*::: -PyParser_SimpleParseFileFlags:FILE*:fp:: -PyParser_SimpleParseFileFlags:const char*:filename:: -PyParser_SimpleParseFileFlags:int:start:: -PyParser_SimpleParseFileFlags:int:flags:: - -PyParser_SimpleParseString:struct _node*::: -PyParser_SimpleParseString:const char*:str:: -PyParser_SimpleParseString:int:start:: - -PyParser_SimpleParseStringFlags:struct _node*::: -PyParser_SimpleParseStringFlags:const char*:str:: -PyParser_SimpleParseStringFlags:int:start:: -PyParser_SimpleParseStringFlags:int:flags:: - -PyParser_SimpleParseStringFlagsFilename:struct _node*::: -PyParser_SimpleParseStringFlagsFilename:const char*:str:: -PyParser_SimpleParseStringFlagsFilename:const char*:filename:: -PyParser_SimpleParseStringFlagsFilename:int:start:: -PyParser_SimpleParseStringFlagsFilename:int:flags:: - PyRun_AnyFile:int::: PyRun_AnyFile:FILE*:fp:: PyRun_AnyFile:const char*:filename:: diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index aecb56eaa4fd2f..3379e41d9de072 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -275,39 +275,8 @@ for more hints. However sometimes you have to run the embedded Python interpreter in the same thread as your rest application and you can't allow the -:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input. The one -solution then is to call :c:func:`PyParser_ParseString` and test for ``e.error`` -equal to ``E_EOF``, which means the input is incomplete. Here's a sample code -fragment, untested, inspired by code from Alex Farber:: - - #define PY_SSIZE_T_CLEAN - #include - #include - #include - #include - #include - #include - - int testcomplete(char *code) - /* code should end in \n */ - /* return -1 for error, 0 for incomplete, 1 for complete */ - { - node *n; - perrdetail e; - - n = PyParser_ParseString(code, &_PyParser_Grammar, - Py_file_input, &e); - if (n == NULL) { - if (e.error == E_EOF) - return 0; - return -1; - } - - PyNode_Free(n); - return 1; - } - -Another solution is trying to compile the received string with +:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input. +A solution is trying to compile the received string with :c:func:`Py_CompileString`. If it compiles without errors, try to execute the returned code object by calling :c:func:`PyEval_EvalCode`. Otherwise save the input for later. If the compilation fails, find out if it's an error or just From 564578bb9abdce188d967e3667da4ef02140dd05 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 22 Jun 2021 17:37:27 +0200 Subject: [PATCH 2/3] Add a blurb and a porting guide to What's New --- Doc/whatsnew/3.10.rst | 41 +++++++++++++++++-- .../2021-06-22-17-00-06.bpo-40939.CGB0I5.rst | 1 + 2 files changed, 39 insertions(+), 3 deletions(-) create mode 100644 Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 9b9dd31a3beabf..f1dbe69eaccff1 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1662,9 +1662,9 @@ Removed that were only being used by the old parser, including ``node.h``, ``parser.h``, ``graminit.h`` and ``grammar.h``. -* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`, - :c:func:`PyParser_SimpleParseStringFlagsFilename`, - :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile` +* Removed the Public C API functions ``PyParser_SimpleParseStringFlags``, + ``PyParser_SimpleParseStringFlagsFilename``, + ``PyParser_SimpleParseFileFlags`` and ``PyNode_Compile`` that were deprecated in 3.9 due to the switch to the new PEG parser. * Removed the ``formatter`` module, which was deprecated in Python 3.4. @@ -1778,6 +1778,41 @@ Changes in the Python API also inherits the current builtins. (Contributed by Victor Stinner in :issue:`42990`.) +Changes in the C API +-------------------- + +* The C API functions ``PyParser_SimpleParseStringFlags``, + ``PyParser_SimpleParseStringFlagsFilename``, + ``PyParser_SimpleParseFileFlags``, ``PyNode_Compile`` and the type + used by these functions, ``struct _node``, were removed due to the switch + to the new PEG parser. + + Source should be now be compiled directly to a code object using, for + example, :c:func:`Py_CompileString`. The resulting code object can then be + evaluated using, for example, :c:func:`PyEval_EvalCode`. + + Specifically: + + * A call to ``PyParser_SimpleParseStringFlags`` followed by + ``PyNode_Compile`` can be replaced by calling :c:func:`Py_CompileString`. + + * There is no direct replacement for ``PyParser_SimpleParseFileFlags``. + To compile code from a `FILE*` argument, you will need to read + the file in C and pass the resulting buffer to :c:func:`Py_CompileString`. + + * To compile a file given a `char *` filename, explicitly open the file, read + it and compile the result. One way to do this is using the :py:mod:`io` + module with :c:func:`PyImport_ImportModule`, :c:func:`PyObject_CallMethod`, + :c:func:`PyBytes_AsString` and :c:func:`Py_CompileString`, + as sketched below. (Declarations and error handling are omitted.) :: + + io_module = Import_ImportModule("io"); + fileobject = PyObject_CallMethod(io_module, "open", "ss", filename, "rb"); + source_bytes_object = PyObject_CallMethod(fileobject, "read", ""); + result = PyObject_CallMethod(fileobject, "close", ""); + source_buf = PyBytes_AsString(source_bytes_object); + code = Py_CompileString(source_buf, filename, Py_file_input); + CPython bytecode changes ======================== diff --git a/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst b/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst new file mode 100644 index 00000000000000..2531ac1ba3c12c --- /dev/null +++ b/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst @@ -0,0 +1 @@ +Removed documentation for the removed ``PyParser_*`` C API. From 7fb594d5138db5ef69fbc430ef33e9ccd561d256 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 24 Jun 2021 13:35:45 +0200 Subject: [PATCH 3/3] Fix ReST syntax Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> --- Doc/whatsnew/3.10.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index f1dbe69eaccff1..c1b209133d8ad4 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1797,10 +1797,10 @@ Changes in the C API ``PyNode_Compile`` can be replaced by calling :c:func:`Py_CompileString`. * There is no direct replacement for ``PyParser_SimpleParseFileFlags``. - To compile code from a `FILE*` argument, you will need to read + To compile code from a ``FILE *`` argument, you will need to read the file in C and pass the resulting buffer to :c:func:`Py_CompileString`. - * To compile a file given a `char *` filename, explicitly open the file, read + * To compile a file given a ``char *`` filename, explicitly open the file, read it and compile the result. One way to do this is using the :py:mod:`io` module with :c:func:`PyImport_ImportModule`, :c:func:`PyObject_CallMethod`, :c:func:`PyBytes_AsString` and :c:func:`Py_CompileString`,