Deprecate the unused CodeObject flag CO_NESTED

[iritkatriel]

This flag is not used by the interpreter. However, it is documented (as an implementation detail) in the inspect doc at https://docs.python.org/3.12/library/inspect.html.

We should consider deprecating it.

[carljm]

Looking for third-party uses, I found https://github.com/pytorch/pytorch/blob/cfb9d0d23314fd28be118b6ca280ded55364e71c/torch/utils/data/datapipes/utils/common.py#L110 and https://github.com/catboost/catboost/blob/dfc3f04050ef9acc3f070f38fe363fec93a3bc8d/contrib/python/joblib/joblib/externals/loky/cloudpickle_wrapper.py#L70

Both appear to be detecting nested functions in order to avoid trying to pickle them, so checking for freevars wouldn't be a suitable alternative in these cases. Checking for "<locals>" in fn.__qualname__ would be.

[hugovk]

Using Victor's https://github.com/vstinner/misc/blob/main/cpython/search_pypi_top.py on the top 5k PyPI packages as of 2022-06-12 (or rather the 4,768 that have sdists), it found 90 matching lines in 13 projects.

They're mostly typeshed definitions.

Full output

$ python ~/github/misc/cpython/search_pypi_top.py . CO_NESTED --quiet
./pydevd-pycharm-222.2889.11.tar.gz: pydevd-pycharm-222.2889.11/_pydev_imps/_pydev_inspect.py: CO_NESTED, CO_GENERATOR, CO_NOFREE = 0x10, 0x20, 0x40
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib64/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv/lib64/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib64/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./hexbytes-0.2.2.tar.gz: hexbytes-0.2.2/venv-py3.8/lib64/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./jedi-0.18.1.tar.gz: jedi-0.18.1/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./jedi-0.18.1.tar.gz: jedi-0.18.1/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu-200/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu-200/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu-200/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-utils-2.0.0.tar.gz: eth-utils-2.0.0/venv-eu-200/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/inspect.pyi: CO_NESTED: int
./coverage-6.4.1.tar.gz: coverage-6.4.1/lab/show_pyc.py: ('CO_NESTED',                   0x00010),
./mypy-0.961.tar.gz: mypy-0.961/mypy/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-abi-ii/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./mypy-0.961.tar.gz: mypy-0.961/mypy/typeshed/stdlib/inspect.pyi: "CO_NESTED",
./mypy-0.961.tar.gz: mypy-0.961/mypy/typeshed/stdlib/inspect.pyi: CO_NESTED: Literal[16]
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-abi-ii/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-abi-ii/lib/python3.9/site-packages/mypy/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-abi-ii/lib/python3.9/site-packages/mypy/typeshed/stdlib/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi/lib/python3.9/site-packages/mypy/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi/lib/python3.9/site-packages/mypy/typeshed/stdlib/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi-3/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi-3/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi-3/lib/python3.9/site-packages/mypy/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth_abi-3.0.0.tar.gz: eth_abi-3.0.0/venv-eabi-3/lib/python3.9/site-packages/mypy/typeshed/stdlib/inspect.pyi: CO_NESTED: int
./pytype-2022.6.6.tar.gz: pytype-2022.6.6/pytype/pyc/loadmarshal.py: CO_NESTED = 0x0010
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./pytype-2022.6.6.tar.gz: pytype-2022.6.6/pytype/typeshed/stdlib/@python2/inspect.pyi: CO_NESTED: int
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./pytype-2022.6.6.tar.gz: pytype-2022.6.6/pytype/typeshed/stdlib/inspect.pyi: "CO_NESTED",
./pytype-2022.6.6.tar.gz: pytype-2022.6.6/pytype/typeshed/stdlib/inspect.pyi: CO_NESTED: Literal[16]
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/.tox/lint/lib/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/.tox/lint/lib/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib64/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv/lib64/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib64/python3.8/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib64/python3.8/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-hash-0.3.2.tar.gz: eth-hash-0.3.2/venv-py3.8/lib64/python3.8/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./joblib-1.1.0.tar.gz: joblib-1.1.0/joblib/externals/loky/cloudpickle_wrapper.py: need_wrap |= getattr(func_code, "co_flags", 0) & inspect.CO_NESTED
./xdis-6.0.4.tar.gz: xdis-6.0.4/xdis/__init__.py: CO_NESTED,
./xdis-6.0.4.tar.gz: xdis-6.0.4/xdis/__init__.py: "CO_NESTED",
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-056/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-056/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-056/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-056/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-060/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-060/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-060/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-060/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-eabi/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-eabi/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-eabi/lib/python3.9/site-packages/mypy/typeshed/stdlib/2/inspect.pyi: CO_NESTED: int
./eth-account-0.6.1.tar.gz: eth-account-0.6.1/venv-ea-eabi/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/inspect.pyi: CO_NESTED: int

Time: 0:00:16.292883
Found 90 matching lines in 13 projects

The non-typeshed results:

$ python ~/github/misc/cpython/search_pypi_top.py . CO_NESTED --quiet | grep -v typeshed
./pydevd-pycharm-222.2889.11.tar.gz: pydevd-pycharm-222.2889.11/_pydev_imps/_pydev_inspect.py: CO_NESTED, CO_GENERATOR, CO_NOFREE = 0x10, 0x20, 0x40
./coverage-6.4.1.tar.gz: coverage-6.4.1/lab/show_pyc.py: ('CO_NESTED',                   0x00010),
./pytype-2022.6.6.tar.gz: pytype-2022.6.6/pytype/pyc/loadmarshal.py: CO_NESTED = 0x0010
./joblib-1.1.0.tar.gz: joblib-1.1.0/joblib/externals/loky/cloudpickle_wrapper.py: need_wrap |= getattr(func_code, "co_flags", 0) & inspect.CO_NESTED
./xdis-6.0.4.tar.gz: xdis-6.0.4/xdis/__init__.py: CO_NESTED,
./xdis-6.0.4.tar.gz: xdis-6.0.4/xdis/__init__.py: "CO_NESTED",

Time: 0:00:15.730672
Found 90 matching lines in 13 projects

[vstinner]

The non-typeshed results

None of these projects set the flag, only cloudpickle_wrapper.py of joblib checks if the flag is set or not.

[vstinner]

This flag is not used by the interpreter.

IMO in this case, it's fine to deprecate the flag. Thanks for checking top 5000 PyPI projects, that's useful to take a decision.

[iritkatriel]

Both appear to be detecting nested functions in order to avoid trying to pickle them, so checking for freevars wouldn't be a suitable alternative in these cases. Checking for "<locals>" in fn.__qualname__ would be.

If people need this, and that's the best alternative then maybe we should keep the flag?

[vstinner]

Oh, right. In Python 3.12, CO_NESTED is still sets properly: it can be used to check if a function was declared inside another function.

import inspect
def f():
    def g(): pass
    return g

def is_nested(func):
    code = func.__code__
    print(f"{func.__name__} is nested?", bool(code.co_flags & inspect.CO_NESTED))

g = f()
is_nested(f)
is_nested(g)

Python 3.12 output:

f is nested? False
g is nested? True

Is there an alternative without testing the CO_NESTED flag?

[carljm]

Is there an alternative without testing the CO_NESTED flag?

I mentioned both alternatives that I know of above. The only real equivalent AFAIK is to check "<locals>" in func.__qualname__, which should give the same result in all cases I'm aware of. A semi-equivalent is to check if the function has freevars, if what you really care about is whether it closes over any values, but some nested functions don't have freevars.

[iritkatriel]

I think we should not remove it. It's used, there is no good alternative, and it's not in anybody's way.