From a6fe738d58a8a9087f3e172a55d600801317d859 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 30 Oct 2021 21:43:19 +0100 Subject: [PATCH 01/28] bpo-45680: Clarify documentation on ``GenericAlias`` objects The documentation on ``GenericAlias`` objects implies at multiple points that only container classes can define ``__class_getitem__``. This is misleading. This PR proposes a rewrite of the documentation to clarify that non-container classes can define ``__class_getitem__``, and to clarify what it means when a non-container class is parameterized. See also: initial discussion of issues with this piece of documentation in #29308, and previous BPO issue [42280](https://bugs.python.org/issue42280). --- Doc/library/stdtypes.rst | 121 +++++++++++++----- .../2021-10-30-21-41-56.bpo-45680.DOLVxK.rst | 3 + 2 files changed, 90 insertions(+), 34 deletions(-) create mode 100644 Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index eea9ddcc140031..846d1981aaa1d1 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4822,35 +4822,66 @@ Generic Alias Type object: GenericAlias pair: Generic; Alias -``GenericAlias`` objects are created by subscripting a class (usually a -container), such as ``list[int]``. They are intended primarily for -:term:`type annotations `. - -Usually, the :ref:`subscription ` of container objects calls the -method :meth:`__getitem__` of the object. However, the subscription of some -containers' classes may call the classmethod :meth:`__class_getitem__` of the -class instead. The classmethod :meth:`__class_getitem__` should return a -``GenericAlias`` object. +``GenericAlias`` objects are generally created by +:ref:`subscripting` a class. They are most often used with +container classes (e.g. :class:`list` or :class:`dict`). They are intended +primarily for use with :term:`type annotations `. For example, +``list[int]`` is a ``GenericAlias`` object created by subscripting the +:class:`list` class with the argument :class:`int`. + +Usually, the :ref:`subscription ` of an object in Python will +call the :meth:`__getitem__` instance method defined on the +object's class. For example, if we have a list +``food = ['spam', 'eggs', 'bacon']``, calling ``food[0]`` will return the same +value as calling ``type(food).__getitem__(food, 0)``. However, if a class +defines the classmethod :meth:`__class_getitem__`, +then the subscription of that class may instead call the class's implementation +of ``__class_getitem__``. This should return a ``GenericAlias`` object if it is +properly defined. For example, because the :class:`list` class defines +``__class_getitem__``, calling ``list[str]`` is equivalent to calling +``list.__class_getitem__(str)``, rather than +``type(list).__getitem__(list, str)``. .. note:: - If the :meth:`__getitem__` of the class' metaclass is present, it will take - precedence over the :meth:`__class_getitem__` defined in the class (see - :pep:`560` for more details). - -The ``GenericAlias`` object acts as a proxy for :term:`generic types -`, implementing *parameterized generics* - a specific instance -of a generic which provides the types for container elements. - -The user-exposed type for the ``GenericAlias`` object can be accessed from -:class:`types.GenericAlias` and used for :func:`isinstance` checks. It can -also be used to create ``GenericAlias`` objects directly. + If :meth:`__getitem__` is defined by a class's + :term:`metaclass`, it will take precedence over a + :meth:`__class_getitem__` classmethod defined by + the class. See :pep:`560` for more details. + +A ``GenericAlias`` object acts as a proxy for a :term:`generic types`, +implementing *parameterized generics*. + +For a container class which implements +:meth:`__class_getitem__`, the argument(s) supplied +to a :ref:`subscription` of the class may indicate the type(s) +of the elements an object contains. For example, ``list[int]`` can be used in +type annotations to signify a :class:`list` in which all the elements are of +type :class:`int`. + +For a class which defines :meth:`__class_getitem__` +but is not a container, the +argument(s) supplied to a subscription of the class will often indicate the +return type(s) of one or more methods defined on an object. For example, +:mod:`regular expressions` can be used on the :class:`str` data type and +the :class:`bytes` data type. If ``x = re.search('foo', 'foo')``, ``x`` will be +a :ref:`re.Match ` object where the return values of +``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can represent +this kind of object in type annotations with the ``GenericAlias`` +``re.Match[str]``. If ``y = re.search(b'bar', b'bar')``, however, ``y`` will +also be an instance of ``re.Match``, but the return values of ``y.group(0)`` +and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would +represent this subtype of :ref:`re.Match ` with +``re.Match[bytes]``. + +``GenericAlias`` objects are instances of the class +:class:`types.GenericAlias`, which can also be used to create ``GenericAlias`` +objects directly. .. describe:: T[X, Y, ...] - Creates a ``GenericAlias`` representing a type ``T`` containing elements - of types *X*, *Y*, and more depending on the ``T`` used. - For example, a function expecting a :class:`list` containing - :class:`float` elements:: + Creates a ``GenericAlias`` representing a type ``T`` parameterized by types + *X*, *Y*, and more depending on the ``T`` used. For example, a function + expecting a :class:`list` containing :class:`float` elements:: def average(values: list[float]) -> float: return sum(values) / len(values) @@ -4900,8 +4931,8 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`__getitem__` method of generics will raise an exception to disallow -mistakes like ``dict[str][str]``:: +The :meth:`__getitem__` method of generics will raise an +exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): @@ -4918,10 +4949,11 @@ in the ``GenericAlias`` object's :attr:`__args__ `. :: dict[str, int] -Standard Generic Collections -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Standard Generic Classes +^^^^^^^^^^^^^^^^^^^^^^^^ -These standard library collections support parameterized generics. +The following standard library classes support parameterized generics. This +list is non-exhaustive. * :class:`tuple` * :class:`list` @@ -4959,12 +4991,31 @@ These standard library collections support parameterized generics. * :class:`collections.abc.ValuesView` * :class:`contextlib.AbstractContextManager` * :class:`contextlib.AbstractAsyncContextManager` +* :class:`dataclasses.Field` +* :class:`functools.cached_property` +* :class:`functools.partialmethod` +* :class:`os.PathLike` +* :class:`pathlib.Path` +* :class:`pathlib.PurePath` +* :class:`pathlib.PurePosixPath` +* :class:`pathlib.PureWindowsPath` +* :class:`queue.LifoQueue` +* :class:`queue.Queue` +* :class:`queue.PriorityQueue` +* :class:`queue.SimpleQueue` * :ref:`re.Pattern ` * :ref:`re.Match ` +* :class:`shelve.Shelf` +* :class:`types.MappingProxyType` +* :class:`weakref.WeakKeyDictionary` +* :class:`weakref.WeakMethod` +* :class:`weakref.WeakSet` +* :class:`weakref.WeakValueDictionary` + -Special Attributes of Generic Alias -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Special Attributes of ``GenericAlias`` objects +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All parameterized generics implement special read-only attributes. @@ -4979,8 +5030,9 @@ All parameterized generics implement special read-only attributes. .. attribute:: genericalias.__args__ This attribute is a :class:`tuple` (possibly of length 1) of generic - types passed to the original :meth:`__class_getitem__` - of the generic container:: + types passed to the original + :meth:`__class_getitem__` of the generic + container:: >>> dict[str, list[int]].__args__ (, list[int]) @@ -5006,7 +5058,8 @@ All parameterized generics implement special read-only attributes. .. seealso:: * :pep:`585` -- "Type Hinting Generics In Standard Collections" - * :meth:`__class_getitem__` -- Used to implement parameterized generics. + * :meth:`__class_getitem__` -- Used to implement + parameterized generics. * :ref:`generics` -- Generics in the :mod:`typing` module. .. versionadded:: 3.9 diff --git a/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst b/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst new file mode 100644 index 00000000000000..79ea6e308e54ac --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst @@ -0,0 +1,3 @@ +Amend the docs on ``GenericAlias`` objects to clarify that non-container +classes can also implement ``__class_getitem__``. Patch contributed by Alex +Waygood. From cef09309ee2d61402c9d2dc01902647c496db390 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 30 Oct 2021 21:52:09 +0100 Subject: [PATCH 02/28] Correct typo --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 846d1981aaa1d1..87aa6cc61c2db0 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4848,7 +4848,7 @@ properly defined. For example, because the :class:`list` class defines :meth:`__class_getitem__` classmethod defined by the class. See :pep:`560` for more details. -A ``GenericAlias`` object acts as a proxy for a :term:`generic types`, +A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. For a container class which implements From 614fce5b61fb99a29cc620a0bddc3baaa57bd756 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sun, 31 Oct 2021 17:57:59 +0000 Subject: [PATCH 03/28] Add more `shelve` classes --- Doc/library/stdtypes.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 87aa6cc61c2db0..062adce7020cdb 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -5005,6 +5005,8 @@ list is non-exhaustive. * :class:`queue.SimpleQueue` * :ref:`re.Pattern ` * :ref:`re.Match ` +* :class:`shelve.BsdDbShelf` +* :class:`shelve.DbfilenameShelf` * :class:`shelve.Shelf` * :class:`types.MappingProxyType` * :class:`weakref.WeakKeyDictionary` From 74269b831aa37b1e5e9b6f4db5191a500f364d51 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 1 Nov 2021 13:23:57 +0000 Subject: [PATCH 04/28] Remove brackets Co-authored-by: Erlend Egeberg Aasland --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 062adce7020cdb..7d98cbbbb60f28 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4824,7 +4824,7 @@ Generic Alias Type ``GenericAlias`` objects are generally created by :ref:`subscripting` a class. They are most often used with -container classes (e.g. :class:`list` or :class:`dict`). They are intended +container classes, such as :class:`list` or :class:`dict`. They are intended primarily for use with :term:`type annotations `. For example, ``list[int]`` is a ``GenericAlias`` object created by subscripting the :class:`list` class with the argument :class:`int`. From 671579bd716909be9017ffcfd93563c8b28826d0 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 1 Nov 2021 17:59:54 +0000 Subject: [PATCH 05/28] Add link to container classes, remove use of the word 'they' --- Doc/library/stdtypes.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 7d98cbbbb60f28..71932138f4459b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4824,10 +4824,11 @@ Generic Alias Type ``GenericAlias`` objects are generally created by :ref:`subscripting` a class. They are most often used with -container classes, such as :class:`list` or :class:`dict`. They are intended -primarily for use with :term:`type annotations `. For example, -``list[int]`` is a ``GenericAlias`` object created by subscripting the -:class:`list` class with the argument :class:`int`. +:ref:`container classes`, such as :class:`list` or +:class:`dict`. For example, ``list[int]`` is a ``GenericAlias`` object created +by subscripting the :class:`list` class with the argument :class:`int`. +``GenericAlias`` objects are intended primarily for use with +:term:`type annotations `. Usually, the :ref:`subscription ` of an object in Python will call the :meth:`__getitem__` instance method defined on the From d367338283c6152ffdf7ff7bf1903d0aa2c3c12e Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 09:51:54 +0000 Subject: [PATCH 06/28] Improve cross-references to data model docs --- Doc/library/stdtypes.rst | 67 +++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 36 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 71932138f4459b..2e1e93f3174cf1 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4831,38 +4831,34 @@ by subscripting the :class:`list` class with the argument :class:`int`. :term:`type annotations `. Usually, the :ref:`subscription ` of an object in Python will -call the :meth:`__getitem__` instance method defined on the -object's class. For example, if we have a list -``food = ['spam', 'eggs', 'bacon']``, calling ``food[0]`` will return the same -value as calling ``type(food).__getitem__(food, 0)``. However, if a class -defines the classmethod :meth:`__class_getitem__`, -then the subscription of that class may instead call the class's implementation -of ``__class_getitem__``. This should return a ``GenericAlias`` object if it is -properly defined. For example, because the :class:`list` class defines -``__class_getitem__``, calling ``list[str]`` is equivalent to calling -``list.__class_getitem__(str)``, rather than -``type(list).__getitem__(list, str)``. +call the :meth:`~object.__getitem__` instance method defined on the object's +class. For example, if we have a list ``food = ['spam', 'eggs', 'bacon']``, +calling ``food[0]`` will return the same value as calling +``type(food).__getitem__(food, 0)``. However, if a class defines the +classmethod :meth:`~object.__class_getitem__`, then the subscription of that +class may instead call the class's implementation of ``__class_getitem__``. +This should return a ``GenericAlias`` object if it is properly defined. For +example, because the :class:`list` class defines ``__class_getitem__``, calling +``list[str]`` is equivalent to calling ``list.__class_getitem__(str)``, rather +than ``type(list).__getitem__(list, str)``. .. note:: - If :meth:`__getitem__` is defined by a class's - :term:`metaclass`, it will take precedence over a - :meth:`__class_getitem__` classmethod defined by - the class. See :pep:`560` for more details. + If :meth:`~object.__getitem__` is defined by a class's :term:`metaclass`, it + will take precedence over a :meth:`~object.__class_getitem__` classmethod + defined by the class. See :pep:`560` for more details. A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. -For a container class which implements -:meth:`__class_getitem__`, the argument(s) supplied -to a :ref:`subscription` of the class may indicate the type(s) -of the elements an object contains. For example, ``list[int]`` can be used in -type annotations to signify a :class:`list` in which all the elements are of -type :class:`int`. - -For a class which defines :meth:`__class_getitem__` -but is not a container, the -argument(s) supplied to a subscription of the class will often indicate the -return type(s) of one or more methods defined on an object. For example, +For a container class which implements :meth:`~object.__class_getitem__`, the +argument(s) supplied to a :ref:`subscription` of the class may +indicate the type(s) of the elements an object contains. For example, +``list[int]`` can be used in type annotations to signify a :class:`list` in +which all the elements are of type :class:`int`. + +For a class which defines :meth:`~object.__class_getitem__` but is not a +container, the argument(s) supplied to a subscription of the class will often +indicate the return type(s) of one or more methods defined on an object. For example, :mod:`regular expressions` can be used on the :class:`str` data type and the :class:`bytes` data type. If ``x = re.search('foo', 'foo')``, ``x`` will be a :ref:`re.Match ` object where the return values of @@ -4932,8 +4928,8 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`__getitem__` method of generics will raise an -exception to disallow mistakes like ``dict[str][str]``:: +The :meth:`~object.__getitem__` method of generics will raise an exception to +disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): @@ -4942,7 +4938,7 @@ exception to disallow mistakes like ``dict[str][str]``:: However, such expressions are valid when :ref:`type variables ` are used. The index must have as many elements as there are type variable items -in the ``GenericAlias`` object's :attr:`__args__ `. :: +in the ``GenericAlias`` object's :attr:`~genericalias.__args__`. :: >>> from typing import TypeVar >>> Y = TypeVar('Y') @@ -5033,9 +5029,8 @@ All parameterized generics implement special read-only attributes. .. attribute:: genericalias.__args__ This attribute is a :class:`tuple` (possibly of length 1) of generic - types passed to the original - :meth:`__class_getitem__` of the generic - container:: + types passed to the original :meth:`~object.__class_getitem__` of the + generic container:: >>> dict[str, list[int]].__args__ (, list[int]) @@ -5054,15 +5049,15 @@ All parameterized generics implement special read-only attributes. .. note:: - A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may not - have correct ``__parameters__`` after substitution because + A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may + not have correct ``__parameters__`` after substitution because :class:`typing.ParamSpec` is intended primarily for static type checking. .. seealso:: * :pep:`585` -- "Type Hinting Generics In Standard Collections" - * :meth:`__class_getitem__` -- Used to implement - parameterized generics. + * :meth:`~object.__class_getitem__` -- Used to implement parameterized + generics. * :ref:`generics` -- Generics in the :mod:`typing` module. .. versionadded:: 3.9 From d945e061e18173eaf1bf8dded736529ad402d5c7 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 11:23:45 +0000 Subject: [PATCH 07/28] Move example to ``__class_getitem__`` docs in the data model --- Doc/library/stdtypes.rst | 36 ++++++++-------------- Doc/reference/datamodel.rst | 60 ++++++++++++++++++++++++++++++++----- 2 files changed, 64 insertions(+), 32 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2e1e93f3174cf1..b8f66ec0b2ec3e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4810,7 +4810,7 @@ Type Annotation Types --- :ref:`Generic Alias `, :ref:`Union single: annotation; type annotation; type hint The core built-in types for :term:`type annotations ` are -:ref:`Generic Alias ` and :ref:`Union `. +:ref:`types-genericalias` and :ref:`types-union`. .. _types-genericalias: @@ -4830,22 +4830,10 @@ by subscripting the :class:`list` class with the argument :class:`int`. ``GenericAlias`` objects are intended primarily for use with :term:`type annotations `. -Usually, the :ref:`subscription ` of an object in Python will -call the :meth:`~object.__getitem__` instance method defined on the object's -class. For example, if we have a list ``food = ['spam', 'eggs', 'bacon']``, -calling ``food[0]`` will return the same value as calling -``type(food).__getitem__(food, 0)``. However, if a class defines the -classmethod :meth:`~object.__class_getitem__`, then the subscription of that -class may instead call the class's implementation of ``__class_getitem__``. -This should return a ``GenericAlias`` object if it is properly defined. For -example, because the :class:`list` class defines ``__class_getitem__``, calling -``list[str]`` is equivalent to calling ``list.__class_getitem__(str)``, rather -than ``type(list).__getitem__(list, str)``. - .. note:: - If :meth:`~object.__getitem__` is defined by a class's :term:`metaclass`, it - will take precedence over a :meth:`~object.__class_getitem__` classmethod - defined by the class. See :pep:`560` for more details. + + It is generally only possible to subscript a class if the class implements + the special method :meth:`~object.__class_getitem__` A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. @@ -4853,17 +4841,17 @@ implementing *parameterized generics*. For a container class which implements :meth:`~object.__class_getitem__`, the argument(s) supplied to a :ref:`subscription` of the class may indicate the type(s) of the elements an object contains. For example, -``list[int]`` can be used in type annotations to signify a :class:`list` in -which all the elements are of type :class:`int`. +``set[bytes]`` can be used in type annotations to signify a :class:`set` in +which all the elements are of type :class:`bytes`. For a class which defines :meth:`~object.__class_getitem__` but is not a container, the argument(s) supplied to a subscription of the class will often -indicate the return type(s) of one or more methods defined on an object. For example, -:mod:`regular expressions` can be used on the :class:`str` data type and -the :class:`bytes` data type. If ``x = re.search('foo', 'foo')``, ``x`` will be -a :ref:`re.Match ` object where the return values of -``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can represent -this kind of object in type annotations with the ``GenericAlias`` +indicate the return type(s) of one or more methods defined on an object. For +example, :mod:`regular expressions` can be used on the :class:`str` data +type and the :class:`bytes` data type. If ``x = re.search('foo', 'foo')``, +``x`` will be a :ref:`re.Match ` object where the return values +of ``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can +represent this kind of object in type annotations with the ``GenericAlias`` ``re.Match[str]``. If ``y = re.search(b'bar', b'bar')``, however, ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index a6eee22fa332cd..419d328429a975 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2215,22 +2215,66 @@ case the instance is itself a class. Emulating generic types ----------------------- -One can implement the generic class syntax as specified by :pep:`484` -(for example ``List[int]``) by defining a special method: +When using :term:`type annotations`, it is often useful to +*parameterize* a generic type using Python's square-brackets notation. +For example, the annotation ``list[int]`` might be used to signify a +:class:`list` in which all the elements are of type :class:`int`. + +.. seealso:: + + :pep:`484` - Type Hints + Introducing Python's framework for type annotations + + :ref:`Generic Alias Types` + Documentation for objects representing parameterized generic classes + + Documentation for the :mod:`typing` module + +A class can generally only be parameterized if it defines the special +classmethod ``__class_getitem__()``. .. classmethod:: object.__class_getitem__(cls, key) Return an object representing the specialization of a generic class by type arguments found in *key*. -This method is looked up on the class object itself, and when defined in -the class body, this method is implicitly a class method. Note, this -mechanism is primarily reserved for use with static type hints, other usage -is discouraged. +.. note:: -.. seealso:: + Usually, the :ref:`subscription ` of an object in Python + using the square-brackets notation will call the :meth:`~object.__getitem__` + instance method defined on the object's class. - :pep:`560` - Core support for typing module and generic types + For example, if we have a list ``food`` as follows:: + + food = ['spam', 'eggs', 'bacon'] + + Calling ``food[0]`` will return the same value as calling:: + + type(food).__getitem__(food, 0) + + However, if a class defines the classmethod ``__class_getitem__()``, then + the subscription of that class may call the class's implementation of + ``__class_getitem__()`` rather than :meth:`~object.__getitem__`. + ``__class_getitem__()`` should return a + :ref:`GenericAlias` object if it is properly defined. + + For example, because the :class:`list` class defines + ``__class_getitem__()``, calling ``list[str]`` is equivalent to calling:: + + list.__class_getitem__(str) + + rather than:: + + type(list).__getitem__(list, str) + +.. note:: + If :meth:`~object.__getitem__` is defined by a class's :term:`metaclass`, it + will take precedence over a ``__class_getitem__()`` classmethod + defined by the class. See :pep:`560` for more details. + +.. note:: + ``__class_getitem__()`` is designed for use with static type hints. Other + usage of this special method is discouraged. .. _callable-types: From 629a29d58ccaae0c7e63fc75e38ab4ae222afe01 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 11:29:26 +0000 Subject: [PATCH 08/28] revert line 4813 --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index b8f66ec0b2ec3e..17628105f3f57f 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4810,7 +4810,7 @@ Type Annotation Types --- :ref:`Generic Alias `, :ref:`Union single: annotation; type annotation; type hint The core built-in types for :term:`type annotations ` are -:ref:`types-genericalias` and :ref:`types-union`. +:ref:`Generic Alias ` and :ref:`Union `. .. _types-genericalias: From 5dd309c84951ade26dfb681990eaa210f66849d3 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 13:04:07 +0000 Subject: [PATCH 09/28] More instances where it is implied that only containers can be parameterised --- Doc/library/stdtypes.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 17628105f3f57f..ded18ac384b13e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4889,9 +4889,9 @@ The builtin functions :func:`isinstance` and :func:`issubclass` do not accept The Python runtime does not enforce :term:`type annotations `. This extends to generic types and their type parameters. When creating -an object from a ``GenericAlias``, container elements are not checked -against their type. For example, the following code is discouraged, but will -run without errors:: +a container object from a ``GenericAlias``, the elements in the container are +not checked against their type. For example, the following code is discouraged, +but will run without errors:: >>> t = list[str] >>> t([1, 2, 3]) @@ -5018,7 +5018,7 @@ All parameterized generics implement special read-only attributes. This attribute is a :class:`tuple` (possibly of length 1) of generic types passed to the original :meth:`~object.__class_getitem__` of the - generic container:: + generic class:: >>> dict[str, list[int]].__args__ (, list[int]) From 5dbccada0cdffc254e9abcb546e800d975ee685c Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 13:38:16 +0000 Subject: [PATCH 10/28] Improve references in glossary and typing docs --- Doc/glossary.rst | 9 +++++---- Doc/library/stdtypes.rst | 4 ++-- Doc/library/typing.rst | 4 ++-- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 1f14946fa13427..9506346cb156be 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -510,12 +510,13 @@ Glossary :func:`functools.singledispatch` decorator, and :pep:`443`. generic type - A :term:`type` that can be parameterized; typically a container like - :class:`list`. Used for :term:`type hints ` and + A :term:`type` that can be parameterized; typically a + :ref:`container class` such as :class:`list` or + :class:`dict`. Used for :term:`type hints ` and :term:`annotations `. - See :pep:`483` for more details, and :mod:`typing` or - :ref:`generic alias type ` for its uses. + For more details, see :ref:`generic alias types`, + :pep:`483`, :pep:`484`, :pep:`585` and the :mod:`typing` module. GIL See :term:`global interpreter lock`. diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index ded18ac384b13e..1f922f02ec9a11 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4916,8 +4916,8 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`~object.__getitem__` method of generics will raise an exception to -disallow mistakes like ``dict[str][str]``:: +The :meth:`~object.__getitem__` method of generic containers will raise an +exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index eb95af378d45fd..76f8f4ae289f72 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -49,8 +49,8 @@ annotations. These include: *Introducing* :class:`Protocol` and the :func:`@runtime_checkable` decorator * :pep:`585`: Type Hinting Generics In Standard Collections - *Introducing* the ability to use builtin collections and ABCs as - :term:`generic types` + *Introducing* :class:`types.GenericAlias` and the ability to use standard + library classes as :ref:`generic types` * :pep:`586`: Literal Types *Introducing* :data:`Literal` * :pep:`589`: TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys From bc457dd532559bc546b650fe5e2bab4e1e2f4d07 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 15:01:48 +0000 Subject: [PATCH 11/28] Fix typing docs link to ``__class_getitem__`` method --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 76f8f4ae289f72..c67f5ba2ff01d1 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -291,7 +291,7 @@ A user-defined class can be defined as a generic class. single type parameter ``T`` . This also makes ``T`` valid as a type within the class body. -The :class:`Generic` base class defines :meth:`__class_getitem__` so that +The :class:`Generic` base class defines :meth:`~object.__class_getitem__` so that ``LoggedVar[t]`` is valid as a type:: from collections.abc import Iterable From d39897121918484c4cc838c29e95f6fd75d8594e Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 2 Nov 2021 17:45:26 +0000 Subject: [PATCH 12/28] Add note about ``typing.Generic`` --- Doc/reference/datamodel.rst | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 419d328429a975..2100db80b47f2f 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2228,7 +2228,9 @@ For example, the annotation ``list[int]`` might be used to signify a :ref:`Generic Alias Types` Documentation for objects representing parameterized generic classes - Documentation for the :mod:`typing` module + :class:`typing.Generic` + Inherit from :class:`typing.Generic` to implement generic classes that + can be parameterized at runtime and understood by static type-checkers. A class can generally only be parameterized if it defines the special classmethod ``__class_getitem__()``. @@ -2238,7 +2240,7 @@ classmethod ``__class_getitem__()``. Return an object representing the specialization of a generic class by type arguments found in *key*. -.. note:: +``__getitem__`` *versus* ``__class_getitem__`` Usually, the :ref:`subscription ` of an object in Python using the square-brackets notation will call the :meth:`~object.__getitem__` @@ -2273,9 +2275,20 @@ classmethod ``__class_getitem__()``. defined by the class. See :pep:`560` for more details. .. note:: - ``__class_getitem__()`` is designed for use with static type hints. Other - usage of this special method is discouraged. + ``__class_getitem__()`` was introduced to implement runtime parameterization + of standard-library generic classes in order to more easily apply + :term:`type-hints` to these classes. + + To implement custom generic classes that can be parameterized at runtime and + understood by static type-checkers, users should either inherit from a + standard library class that already implements ``__class_getitem__()``, or + inherit from :class:`typing.Generic`, which has its own implementation of + ``__class_getitem__()``. + Custom implementations of ``__class_getitem__()`` on classes defined outside + of the standard library may not be understood by third-party type-checkers + such as mypy. Using ``__class_getitem__()`` on any class for purposes other + than type-hinting is discouraged. .. _callable-types: @@ -2388,6 +2401,12 @@ through the object's keys; for sequences, it should iterate through the values. :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal indexes to allow proper detection of the end of the sequence. + .. note:: + + When :ref:`subscripting` a *class*, the special + classmethod :meth:`~object.__class_getitem__` may be called instead of + ``__getitem__()``. + .. method:: object.__setitem__(self, key, value) From af257a624f2e40faec5c9ab49bf9684f79452500 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 18 Nov 2021 23:14:37 +0000 Subject: [PATCH 13/28] Update glossary --- Doc/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 9506346cb156be..ccbfc0e6c36c7d 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -516,7 +516,7 @@ Glossary :term:`annotations `. For more details, see :ref:`generic alias types`, - :pep:`483`, :pep:`484`, :pep:`585` and the :mod:`typing` module. + :pep:`483`, :pep:`484`, :pep:`585`, and the :mod:`typing` module. GIL See :term:`global interpreter lock`. From eb7e7becfcc87ac09f77d8e3efdff27ef896599e Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 18 Nov 2021 23:15:42 +0000 Subject: [PATCH 14/28] Update typing --- Doc/library/typing.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c67f5ba2ff01d1..735d477db4371e 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -18,10 +18,9 @@ -------------- This module provides runtime support for type hints. The most fundamental -support consists of the types :data:`Any`, :data:`Union`, :data:`Tuple`, -:data:`Callable`, :class:`TypeVar`, and :class:`Generic`. For a full -specification, please see :pep:`484`. For a simplified introduction to type -hints, see :pep:`483`. +support consists of the types :data:`Any`, :data:`Union`, :data:`Callable`, +:class:`TypeVar`, and :class:`Generic`. For a full specification, please see +:pep:`484`. For a simplified introduction to type hints, see :pep:`483`. The function below takes and returns a string and is annotated as follows:: @@ -257,6 +256,7 @@ called :class:`TypeVar`. def first(l: Sequence[T]) -> T: # Generic function return l[0] +.. _user-defined-generics: User-defined generic types ========================== @@ -291,8 +291,8 @@ A user-defined class can be defined as a generic class. single type parameter ``T`` . This also makes ``T`` valid as a type within the class body. -The :class:`Generic` base class defines :meth:`~object.__class_getitem__` so that -``LoggedVar[t]`` is valid as a type:: +The :class:`Generic` base class defines :meth:`~object.__class_getitem__` so +that ``LoggedVar[t]`` is valid as a type:: from collections.abc import Iterable From 5c8fe0d88efec4914361a13404b8aaaded4832cd Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 18 Nov 2021 23:16:42 +0000 Subject: [PATCH 15/28] update datamodel --- Doc/reference/datamodel.rst | 171 +++++++++++++++++++++++++----------- 1 file changed, 118 insertions(+), 53 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 2100db80b47f2f..1ecfa81e3b3363 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2216,7 +2216,7 @@ Emulating generic types ----------------------- When using :term:`type annotations`, it is often useful to -*parameterize* a generic type using Python's square-brackets notation. +*parameterize* a :term:`generic type` using Python's square-brackets notation. For example, the annotation ``list[int]`` might be used to signify a :class:`list` in which all the elements are of type :class:`int`. @@ -2228,67 +2228,130 @@ For example, the annotation ``list[int]`` might be used to signify a :ref:`Generic Alias Types` Documentation for objects representing parameterized generic classes - :class:`typing.Generic` - Inherit from :class:`typing.Generic` to implement generic classes that - can be parameterized at runtime and understood by static type-checkers. + :ref:`Generics`, :ref:`user-defined generics` and :class:`typing.Generic` + Documentation on how to implement generic classes that can be + parameterized at runtime and understood by static type-checkers. -A class can generally only be parameterized if it defines the special -classmethod ``__class_getitem__()``. +A class can *generally* only be parameterized if it defines the special +class method ``__class_getitem__()``. .. classmethod:: object.__class_getitem__(cls, key) Return an object representing the specialization of a generic class by type arguments found in *key*. -``__getitem__`` *versus* ``__class_getitem__`` + When defined on a class, ``__class_getitem__()`` is automatically a class + method. As such, there is no need for it to be decorated with + :func:`@classmethod` when it is defined. - Usually, the :ref:`subscription ` of an object in Python - using the square-brackets notation will call the :meth:`~object.__getitem__` - instance method defined on the object's class. - For example, if we have a list ``food`` as follows:: +The purpose of *__class_getitem__* +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - food = ['spam', 'eggs', 'bacon'] +The purpose of :meth:`~object.__class_getitem__` is to allow runtime +parameterization of standard-library generic classes in order to more easily +apply :term:`type hints` to these classes. - Calling ``food[0]`` will return the same value as calling:: +To implement custom generic classes that can be parameterized at runtime and +understood by static type-checkers, users should either inherit from a standard +library class that already implements :meth:`~object.__class_getitem__`, or +inherit from :class:`typing.Generic`, which has its own implementation of +``__class_getitem__()``. - type(food).__getitem__(food, 0) +Custom implementations of :meth:`~object.__class_getitem__` on classes defined +outside of the standard library may not be understood by third-party +type-checkers such as mypy. Using ``__class_getitem__()`` on any class for +purposes other than type hinting is discouraged. - However, if a class defines the classmethod ``__class_getitem__()``, then - the subscription of that class may call the class's implementation of - ``__class_getitem__()`` rather than :meth:`~object.__getitem__`. - ``__class_getitem__()`` should return a - :ref:`GenericAlias` object if it is properly defined. - For example, because the :class:`list` class defines - ``__class_getitem__()``, calling ``list[str]`` is equivalent to calling:: +.. _classgetitem-versus-getitem: - list.__class_getitem__(str) - rather than:: +*__class_getitem__* versus *__getitem__* +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - type(list).__getitem__(list, str) +Usually, the :ref:`subscription` of an object using square +brackets will call the :meth:`~object.__getitem__` instance method defined on +the object's class. However, if the object being subscribed is itself a class, +the class method :meth:`~object.__class_getitem__` may be called instead. +``__class_getitem__()`` should return a :ref:`GenericAlias` +object if it is properly defined. -.. note:: - If :meth:`~object.__getitem__` is defined by a class's :term:`metaclass`, it - will take precedence over a ``__class_getitem__()`` classmethod - defined by the class. See :pep:`560` for more details. +Presented with the :term:`expression` ``obj[x]``, the Python interpreter +follows something like the following process to decide whether +:meth:`~object.__getitem__` or :meth:`~object.__class_getitem__` should be +called:: -.. note:: - ``__class_getitem__()`` was introduced to implement runtime parameterization - of standard-library generic classes in order to more easily apply - :term:`type-hints` to these classes. + from inspect import isclass + + def subscribe(obj, x): + """Return the result of the expression `obj[x]`""" + + class_of_obj = type(obj) + + # If the class of obj defines __getitem__, + # call class_of_obj.__getitem__(obj, x) + if hasattr(class_of_obj, '__getitem__'): + return class_of_obj.__getitem__(obj, x) + + # Else, if obj is a class and defines __class_getitem__, + # call obj.__class_getitem__(x) + elif isclass(obj) and hasattr(obj, '__class_getitem__'): + return obj.__class_getitem__(x) + + # Else, raise an exception + else: + raise TypeError( + f"'{class_of_obj.__name__}' object is not subscriptable" + ) - To implement custom generic classes that can be parameterized at runtime and - understood by static type-checkers, users should either inherit from a - standard library class that already implements ``__class_getitem__()``, or - inherit from :class:`typing.Generic`, which has its own implementation of - ``__class_getitem__()``. +In Python, all classes are themselves instances of other classes. The class of +a class is known as that class's :term:`metaclass`, and most classes have the +:class:`type` class as their metaclass. :class:`type` does not define +:meth:`~object.__getitem__`, meaning that expressions such as ``list[int]``, +``dict[str, float]`` and ``tuple[str, bytes]`` all result in +:meth:`~object.__class_getitem__` being called:: + + >>> # list has class "type" as its metaclass, like most classes: + >>> type(list) + + >>> type(dict) == type(list) == type(tuple) == type(str) == type(bytes) + True + >>> # "list[int]" calls "list.__class_getitem__(int)" + >>> list[int] + list[int] + >>> # list.__class_getitem__ returns a GenericAlias object: + >>> type(list[int]) + + +However, if a class has a custom metaclass that defines +:meth:`~object.__getitem__`, subscribing the class may result in different +behaviour. An example of this can be found in the :mod:`enum` module:: + + >>> from enum import Enum + >>> class Menu(Enum): + ... """A breakfast menu""" + ... SPAM = 'spam' + ... BACON = 'bacon' + ... + >>> # Enum classes have a custom metaclass: + >>> type(Menu) + + >>> # EnumMeta defines __getitem__, + >>> # so __class_getitem__ is not called, + >>> # and the result is not a GenericAlias object: + >>> Menu['SPAM'] + + >>> type(Menu['SPAM']) + + + +.. seealso:: + :pep:`560` - Core Support for typing module and generic types + Introducing :meth:`~object.__class_getitem__`, and outlining when a + :ref:`subscription` results in ``__class_getitem__()`` + being called instead of :meth:`~object.__getitem__` - Custom implementations of ``__class_getitem__()`` on classes defined outside - of the standard library may not be understood by third-party type-checkers - such as mypy. Using ``__class_getitem__()`` on any class for purposes other - than type-hinting is discouraged. .. _callable-types: @@ -2387,25 +2450,27 @@ through the object's keys; for sequences, it should iterate through the values. .. method:: object.__getitem__(self, key) - Called to implement evaluation of ``self[key]``. For sequence types, the - accepted keys should be integers and slice objects. Note that the special - interpretation of negative indexes (if the class wishes to emulate a sequence - type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate - type, :exc:`TypeError` may be raised; if of a value outside the set of indexes - for the sequence (after any special interpretation of negative values), - :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not - in the container), :exc:`KeyError` should be raised. + Called to implement evaluation of ``self[key]``. For :term:`sequence` types, + the accepted keys should be integers and slice objects. Note that the + special interpretation of negative indexes (if the class wishes to emulate a + :term:`sequence` type) is up to the :meth:`__getitem__` method. If *key* is + of an inappropriate type, :exc:`TypeError` may be raised; if of a value + outside the set of indexes for the sequence (after any special + interpretation of negative values), :exc:`IndexError` should be raised. For + :term:`mapping` types, if *key* is missing (not in the container), + :exc:`KeyError` should be raised. .. note:: - :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal - indexes to allow proper detection of the end of the sequence. + :keyword:`for` loops expect that an :exc:`IndexError` will be raised for + illegal indexes to allow proper detection of the end of the sequence. .. note:: When :ref:`subscripting` a *class*, the special - classmethod :meth:`~object.__class_getitem__` may be called instead of - ``__getitem__()``. + class method :meth:`~object.__class_getitem__` may be called instead of + ``__getitem__()``. See :ref:`classgetitem-versus-getitem` for more + details. .. method:: object.__setitem__(self, key, value) From 010ca866525733e4cb45274048bed4bc22dae9dd Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 18 Nov 2021 23:18:49 +0000 Subject: [PATCH 16/28] delete news --- .../Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst diff --git a/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst b/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst deleted file mode 100644 index 79ea6e308e54ac..00000000000000 --- a/Misc/NEWS.d/next/Documentation/2021-10-30-21-41-56.bpo-45680.DOLVxK.rst +++ /dev/null @@ -1,3 +0,0 @@ -Amend the docs on ``GenericAlias`` objects to clarify that non-container -classes can also implement ``__class_getitem__``. Patch contributed by Alex -Waygood. From 92e046394cfe6c475048d0c62a85143d73392b28 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 19 Nov 2021 00:03:40 +0000 Subject: [PATCH 17/28] A few tweaks --- Doc/library/stdtypes.rst | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 1f922f02ec9a11..43ef3483bd076e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4848,15 +4848,18 @@ For a class which defines :meth:`~object.__class_getitem__` but is not a container, the argument(s) supplied to a subscription of the class will often indicate the return type(s) of one or more methods defined on an object. For example, :mod:`regular expressions` can be used on the :class:`str` data -type and the :class:`bytes` data type. If ``x = re.search('foo', 'foo')``, -``x`` will be a :ref:`re.Match ` object where the return values -of ``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can -represent this kind of object in type annotations with the ``GenericAlias`` -``re.Match[str]``. If ``y = re.search(b'bar', b'bar')``, however, ``y`` will -also be an instance of ``re.Match``, but the return values of ``y.group(0)`` -and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would -represent this subtype of :ref:`re.Match ` with -``re.Match[bytes]``. +type and the :class:`bytes` data type: + +* If ``x = re.search('foo', 'foo')``, ``x`` will be a + :ref:`re.Match ` object where the return values of + ``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can + represent this kind of object in type annotations with the ``GenericAlias`` + ``re.Match[str]`` + +* If ``y = re.search(b'bar', b'bar')``, however, ``y`` will also be an instance + of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will + both be of type :class:`bytes`. In type annotations, we would represent this + subtype of :ref:`re.Match ` with ``re.Match[bytes]``. ``GenericAlias`` objects are instances of the class :class:`types.GenericAlias`, which can also be used to create ``GenericAlias`` @@ -5043,10 +5046,17 @@ All parameterized generics implement special read-only attributes. .. seealso:: - * :pep:`585` -- "Type Hinting Generics In Standard Collections" - * :meth:`~object.__class_getitem__` -- Used to implement parameterized - generics. - * :ref:`generics` -- Generics in the :mod:`typing` module. + :pep:`484` - Type Hints + Introducing Python's framework for type annotations + + :pep:`585` - "Type Hinting Generics In Standard Collections" + Introducing the ability to natively parameterize standard-library + classes, provided they implement the special class method + :meth:`~object.__class_getitem__`. + + :ref:`Generics`, :ref:`user-defined generics` and :class:`typing.Generic` + Documentation on how to implement generic classes that can be + parameterized at runtime and understood by static type-checkers. .. versionadded:: 3.9 From dd3369ec6a0bc229eecda7b58b0c2bb378d2f2c6 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 19 Nov 2021 15:07:36 +0000 Subject: [PATCH 18/28] Add full stop. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 43ef3483bd076e..80bd1ff9aa1f49 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4833,7 +4833,7 @@ by subscripting the :class:`list` class with the argument :class:`int`. .. note:: It is generally only possible to subscript a class if the class implements - the special method :meth:`~object.__class_getitem__` + the special method :meth:`~object.__class_getitem__`. A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. From 6e2d8f58ef66d559236f47b10179ba383c0c2cb3 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 19 Nov 2021 15:07:55 +0000 Subject: [PATCH 19/28] Remove redundant clause. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 80bd1ff9aa1f49..2c1066682b2688 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4838,7 +4838,7 @@ by subscripting the :class:`list` class with the argument :class:`int`. A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. -For a container class which implements :meth:`~object.__class_getitem__`, the +For a container class, the argument(s) supplied to a :ref:`subscription` of the class may indicate the type(s) of the elements an object contains. For example, ``set[bytes]`` can be used in type annotations to signify a :class:`set` in From 43f9429f88a238281989810d2958449f4de9f53d Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 19 Nov 2021 15:08:09 +0000 Subject: [PATCH 20/28] Update Doc/library/stdtypes.rst Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2c1066682b2688..f1d6cdf76b892d 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4847,7 +4847,7 @@ which all the elements are of type :class:`bytes`. For a class which defines :meth:`~object.__class_getitem__` but is not a container, the argument(s) supplied to a subscription of the class will often indicate the return type(s) of one or more methods defined on an object. For -example, :mod:`regular expressions` can be used on the :class:`str` data +example, :mod:`regular expressions` can be used on both the :class:`str` data type and the :class:`bytes` data type: * If ``x = re.search('foo', 'foo')``, ``x`` will be a From ba2af17d4fcdba4b38cba33142e0867e60ccb3a7 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 29 Nov 2021 01:33:05 +0000 Subject: [PATCH 21/28] Reduce diff --- Doc/library/stdtypes.rst | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index f1d6cdf76b892d..5e90dce31c60a1 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4868,8 +4868,9 @@ objects directly. .. describe:: T[X, Y, ...] Creates a ``GenericAlias`` representing a type ``T`` parameterized by types - *X*, *Y*, and more depending on the ``T`` used. For example, a function - expecting a :class:`list` containing :class:`float` elements:: + *X*, *Y*, and more depending on the ``T`` used. + For example, a function expecting a :class:`list` containing + :class:`float` elements:: def average(values: list[float]) -> float: return sum(values) / len(values) @@ -4892,9 +4893,9 @@ The builtin functions :func:`isinstance` and :func:`issubclass` do not accept The Python runtime does not enforce :term:`type annotations `. This extends to generic types and their type parameters. When creating -a container object from a ``GenericAlias``, the elements in the container are -not checked against their type. For example, the following code is discouraged, -but will run without errors:: +a container object from a ``GenericAlias``, the elements in the container are not checked +against their type. For example, the following code is discouraged, but will +run without errors:: >>> t = list[str] >>> t([1, 2, 3]) @@ -4919,8 +4920,8 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`~object.__getitem__` method of generic containers will raise an -exception to disallow mistakes like ``dict[str][str]``:: +The :meth:`~object.__getitem__` method of generic containers will raise an exception to disallow +mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): @@ -5040,8 +5041,8 @@ All parameterized generics implement special read-only attributes. .. note:: - A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may - not have correct ``__parameters__`` after substitution because + A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may not + have correct ``__parameters__`` after substitution because :class:`typing.ParamSpec` is intended primarily for static type checking. .. seealso:: From 2776b8d63b23e34db70bec2a86bf796b84c43441 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 29 Nov 2021 01:35:15 +0000 Subject: [PATCH 22/28] Trailing whitespace --- Doc/library/stdtypes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 5e90dce31c60a1..afe39c821087bc 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4893,7 +4893,7 @@ The builtin functions :func:`isinstance` and :func:`issubclass` do not accept The Python runtime does not enforce :term:`type annotations `. This extends to generic types and their type parameters. When creating -a container object from a ``GenericAlias``, the elements in the container are not checked +a container object from a ``GenericAlias``, the elements in the container are not checked against their type. For example, the following code is discouraged, but will run without errors:: @@ -4920,7 +4920,7 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`~object.__getitem__` method of generic containers will raise an exception to disallow +The :meth:`~object.__getitem__` method of generic containers will raise an exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] From f2efd8b35b4524ac45cbb124dbde75e37c35af82 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 4 Dec 2021 16:00:03 +0000 Subject: [PATCH 23/28] Update Doc/library/stdtypes.rst Clarify example with `bytes` Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index afe39c821087bc..3d30abf25ace2e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4856,7 +4856,7 @@ type and the :class:`bytes` data type: represent this kind of object in type annotations with the ``GenericAlias`` ``re.Match[str]`` -* If ``y = re.search(b'bar', b'bar')``, however, ``y`` will also be an instance +* If ``y = re.search(b'bar', b'bar')``, (note the `b` for :class:`bytes`), ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would represent this subtype of :ref:`re.Match ` with ``re.Match[bytes]``. From ec8a509824f7fddfad5b895a9d01f3b2bf42aff0 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 4 Dec 2021 16:09:57 +0000 Subject: [PATCH 24/28] Fix CI --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 3d30abf25ace2e..2694578993b174 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4856,7 +4856,7 @@ type and the :class:`bytes` data type: represent this kind of object in type annotations with the ``GenericAlias`` ``re.Match[str]`` -* If ``y = re.search(b'bar', b'bar')``, (note the `b` for :class:`bytes`), ``y`` will also be an instance +* If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would represent this subtype of :ref:`re.Match ` with ``re.Match[bytes]``. From 8fad702aac2c9335e7e5bad4cb5e704551940292 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sun, 5 Dec 2021 13:27:09 +0000 Subject: [PATCH 25/28] Improve markup readability --- Doc/library/stdtypes.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2694578993b174..a92f9e1a344e1a 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4823,8 +4823,8 @@ Generic Alias Type pair: Generic; Alias ``GenericAlias`` objects are generally created by -:ref:`subscripting` a class. They are most often used with -:ref:`container classes`, such as :class:`list` or +:ref:`subscripting ` a class. They are most often used with +:ref:`container classes `, such as :class:`list` or :class:`dict`. For example, ``list[int]`` is a ``GenericAlias`` object created by subscripting the :class:`list` class with the argument :class:`int`. ``GenericAlias`` objects are intended primarily for use with @@ -4839,7 +4839,7 @@ A ``GenericAlias`` object acts as a proxy for a :term:`generic type`, implementing *parameterized generics*. For a container class, the -argument(s) supplied to a :ref:`subscription` of the class may +argument(s) supplied to a :ref:`subscription ` of the class may indicate the type(s) of the elements an object contains. For example, ``set[bytes]`` can be used in type annotations to signify a :class:`set` in which all the elements are of type :class:`bytes`. @@ -4847,7 +4847,7 @@ which all the elements are of type :class:`bytes`. For a class which defines :meth:`~object.__class_getitem__` but is not a container, the argument(s) supplied to a subscription of the class will often indicate the return type(s) of one or more methods defined on an object. For -example, :mod:`regular expressions` can be used on both the :class:`str` data +example, :mod:`regular expressions ` can be used on both the :class:`str` data type and the :class:`bytes` data type: * If ``x = re.search('foo', 'foo')``, ``x`` will be a @@ -5055,7 +5055,7 @@ All parameterized generics implement special read-only attributes. classes, provided they implement the special class method :meth:`~object.__class_getitem__`. - :ref:`Generics`, :ref:`user-defined generics` and :class:`typing.Generic` + :ref:`Generics`, :ref:`user-defined generics ` and :class:`typing.Generic` Documentation on how to implement generic classes that can be parameterized at runtime and understood by static type-checkers. From e5a57e8fa37e5fbbc7d3ae5dee4bed9a399ef0d0 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sun, 5 Dec 2021 13:53:55 +0000 Subject: [PATCH 26/28] Steer clear of the word "subtype" --- Doc/library/stdtypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index a92f9e1a344e1a..90bcd44d1bb2ac 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4859,7 +4859,7 @@ type and the :class:`bytes` data type: * If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would represent this - subtype of :ref:`re.Match ` with ``re.Match[bytes]``. + variety of :ref:`re.Match ` with ``re.Match[bytes]``. ``GenericAlias`` objects are instances of the class :class:`types.GenericAlias`, which can also be used to create ``GenericAlias`` From ae21ffe002e672e89edf32d40b85317fc5c2f0a5 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sun, 5 Dec 2021 14:00:42 +0000 Subject: [PATCH 27/28] Tweak --- Doc/library/stdtypes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 90bcd44d1bb2ac..d38aaec7b5fb6a 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4826,7 +4826,7 @@ Generic Alias Type :ref:`subscripting ` a class. They are most often used with :ref:`container classes `, such as :class:`list` or :class:`dict`. For example, ``list[int]`` is a ``GenericAlias`` object created -by subscripting the :class:`list` class with the argument :class:`int`. +by subscripting the ``list`` class with the argument :class:`int`. ``GenericAlias`` objects are intended primarily for use with :term:`type annotations `. @@ -4859,7 +4859,7 @@ type and the :class:`bytes` data type: * If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), ``y`` will also be an instance of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will both be of type :class:`bytes`. In type annotations, we would represent this - variety of :ref:`re.Match ` with ``re.Match[bytes]``. + variety of :ref:`re.Match ` objects with ``re.Match[bytes]``. ``GenericAlias`` objects are instances of the class :class:`types.GenericAlias`, which can also be used to create ``GenericAlias`` From 3944e8caf8034da4828e9d49bfc2836477cf9d8c Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Wed, 19 Jan 2022 22:22:03 +0800 Subject: [PATCH 28/28] Apply formatting suggestions from code review --- Doc/library/stdtypes.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index d38aaec7b5fb6a..7d1cf0e910ffd0 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4854,11 +4854,12 @@ type and the :class:`bytes` data type: :ref:`re.Match ` object where the return values of ``x.group(0)`` and ``x[0]`` will both be of type :class:`str`. We can represent this kind of object in type annotations with the ``GenericAlias`` - ``re.Match[str]`` + ``re.Match[str]``. -* If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), ``y`` will also be an instance - of ``re.Match``, but the return values of ``y.group(0)`` and ``y[0]`` will - both be of type :class:`bytes`. In type annotations, we would represent this +* If ``y = re.search(b'bar', b'bar')``, (note the ``b`` for :class:`bytes`), + ``y`` will also be an instance of ``re.Match``, but the return + values of ``y.group(0)`` and ``y[0]`` will both be of type + :class:`bytes`. In type annotations, we would represent this variety of :ref:`re.Match ` objects with ``re.Match[bytes]``. ``GenericAlias`` objects are instances of the class @@ -4920,8 +4921,8 @@ Calling :func:`repr` or :func:`str` on a generic shows the parameterized type:: >>> str(list[int]) 'list[int]' -The :meth:`~object.__getitem__` method of generic containers will raise an exception to disallow -mistakes like ``dict[str][str]``:: +The :meth:`~object.__getitem__` method of generic containers will raise an +exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): @@ -5048,7 +5049,7 @@ All parameterized generics implement special read-only attributes. .. seealso:: :pep:`484` - Type Hints - Introducing Python's framework for type annotations + Introducing Python's framework for type annotations. :pep:`585` - "Type Hinting Generics In Standard Collections" Introducing the ability to natively parameterize standard-library