From 30570c9832db9c1216aadb78d039a3c460d6a278 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Mon, 20 Jul 2020 21:48:30 -0300 Subject: [PATCH 01/18] refactored typing.rst, see Issue40979 --- Doc/library/typing.rst | 1203 ++++++++++++++++++++-------------------- 1 file changed, 614 insertions(+), 589 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e5143c5f8d9e4a..d5370988d9c213 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -435,48 +435,175 @@ can define new custom protocols to fully enjoy structural subtyping (see examples below). -Classes, functions, and decorators ----------------------------------- +Module contents +--------------- The module defines the following classes, functions and decorators: -.. class:: TypeVar - Type variable. +Special typing primitives +========================= - Usage:: +.. data:: Annotated - T = TypeVar('T') # Can be anything - A = TypeVar('A', str, bytes) # Must be str or bytes + A type, introduced in :pep:`593` (``Flexible function and variable + annotations``), to decorate existing types with context-specific metadata + (possibly multiple pieces of it, as ``Annotated`` is variadic). + Specifically, a type ``T`` can be annotated with metadata ``x`` via the + typehint ``Annotated[T, x]``. This metadata can be used for either static + analysis or at runtime. If a library (or tool) encounters a typehint + ``Annotated[T, x]`` and has no special logic for metadata ``x``, it + should ignore it and simply treat the type as ``T``. Unlike the + ``no_type_check`` functionality that currently exists in the ``typing`` + module which completely disables typechecking annotations on a function + or a class, the ``Annotated`` type allows for both static typechecking + of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) + together with runtime access to ``x`` within a specific application. - Type variables exist primarily for the benefit of static type - checkers. They serve as the parameters for generic types as well - as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows:: + Ultimately, the responsibility of how to interpret the annotations (if + at all) is the responsibility of the tool or library encountering the + ``Annotated`` type. A tool or library encountering an ``Annotated`` type + can scan through the annotations to determine if they are of interest + (e.g., using ``isinstance()``). - def repeat(x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n + When a tool or a library does not support annotations or encounters an + unknown annotation it should just ignore it and treat annotated type as + the underlying type. - def longest(x: A, y: A) -> A: - """Return the longest of two strings.""" - return x if len(x) >= len(y) else y + It's up to the tool consuming the annotations to decide whether the + client is allowed to have several annotations on one type and how to + merge those annotations. - The latter example's signature is essentially the overloading - of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note - that if the arguments are instances of some subclass of :class:`str`, - the return type is still plain :class:`str`. + Since the ``Annotated`` type allows you to put several annotations of + the same (or different) type(s) on any node, the tools or libraries + consuming those annotations are in charge of dealing with potential + duplicates. For example, if you are doing value range analysis you might + allow this:: - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. + T1 = Annotated[int, ValueRange(-10, 5)] + T2 = Annotated[T1, ValueRange(-20, 3)] - Type variables may be marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. Alternatively, - a type variable may specify an upper bound using ``bound=``. - This means that an actual type substituted (explicitly or implicitly) - for the type variable must be a subclass of the boundary type, - see :pep:`484`. + Passing ``include_extras=True`` to :func:`get_type_hints` lets one + access the extra annotations at runtime. + + The details of the syntax: + + * The first argument to ``Annotated`` must be a valid type + + * Multiple type annotations are supported (``Annotated`` supports variadic + arguments):: + + Annotated[int, ValueRange(3, 10), ctype("char")] + + * ``Annotated`` must be called with at least two arguments ( + ``Annotated[int]`` is not valid) + + * The order of the annotations is preserved and matters for equality + checks:: + + Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ + int, ctype("char"), ValueRange(3, 10) + ] + + * Nested ``Annotated`` types are flattened, with metadata ordered + starting with the innermost annotation:: + + Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ + int, ValueRange(3, 10), ctype("char") + ] + + * Duplicated annotations are not removed:: + + Annotated[int, ValueRange(3, 10)] != Annotated[ + int, ValueRange(3, 10), ValueRange(3, 10) + ] + + * ``Annotated`` can be used with nested and generic aliases:: + + T = TypeVar('T') + Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] + V = Vec[int] + + V == Annotated[List[Tuple[int, int]], MaxLen(10)] + + .. versionadded:: 3.9 + + +.. data:: Any + + Special type indicating an unconstrained type. + + * Every type is compatible with :data:`Any`. + * :data:`Any` is compatible with every type. + +.. data:: Callable + + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. + + The subscription syntax must always be used with exactly two + values: the argument list and the return type. The argument list + must be a list of types or an ellipsis; the return type must be + a single type. + + There is no syntax to indicate optional or keyword arguments; + such function types are rarely used as callback types. + ``Callable[..., ReturnType]`` (literal ellipsis) can be used to + type hint a callable taking any number of arguments and returning + ``ReturnType``. A plain :data:`Callable` is equivalent to + ``Callable[..., Any]``, and in turn to + :class:`collections.abc.Callable`. + +.. data:: ClassVar + + Special type construct to mark class variables. + + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + :data:`ClassVar` accepts only types and cannot be further subscribed. + + :data:`ClassVar` is not a class itself, and should not + be used with :func:`isinstance` or :func:`issubclass`. + :data:`ClassVar` does not change Python runtime behavior, but + it can be used by third-party type checkers. For example, a type checker + might flag the following code as an error:: + + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK + + .. versionadded:: 3.5.3 + +.. data:: Final + + A special typing construct to indicate to type checkers that a name + cannot be re-assigned or overridden in a subclass. For example:: + + MAX_SIZE: Final = 9000 + MAX_SIZE += 1 # Error reported by type checker + + class Connection: + TIMEOUT: Final[int] = 10 + + class FastConnector(Connection): + TIMEOUT = 1 # Error reported by type checker + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 + +.. class:: ForwardRef + + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. .. class:: Generic @@ -502,6 +629,73 @@ The module defines the following classes, functions and decorators: except KeyError: return default +.. data:: Literal + + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent to + the provided literal (or one of several literals). For example:: + + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... + + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... + + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker + + ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value + is allowed as type argument to ``Literal[...]``, but type checkers may + impose restrictions. See :pep:`586` for more details about literal types. + + .. versionadded:: 3.8 + +.. function:: NewType(name, tp) + + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) + + .. versionadded:: 3.5.2 + +.. data:: NoReturn + + Special type indicating that a function never returns. + For example:: + + from typing import NoReturn + + def stop() -> NoReturn: + raise RuntimeError('no way') + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + +.. data:: Optional + + Optional type. + + ``Optional[X]`` is equivalent to ``Union[X, None]``. + + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ``Optional`` qualifier on its type + annotation just because it is optional. For example:: + + def foo(arg: int = 0) -> None: + ... + + On the other hand, if an explicit value of ``None`` is allowed, the + use of ``Optional`` is appropriate, whether the argument is optional + or not. For example:: + + def foo(arg: Optional[int] = None) -> None: + ... + .. class:: Protocol(Generic) Base class for protocol classes. Protocol classes are defined like this:: @@ -577,176 +771,212 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.5.2 -.. class:: Iterable(Generic[T_co]) - - A generic version of :class:`collections.abc.Iterable`. - -.. class:: Iterator(Iterable[T_co]) +.. class:: TypedDict(dict) - A generic version of :class:`collections.abc.Iterator`. + A simple typed namespace. At runtime it is a plain :class:`dict`. -.. class:: Reversible(Iterable[T_co]) + ``TypedDict`` creates a dictionary type that expects all of its + instances to have a certain set of keys, where each key is + associated with a value of a consistent type. This expectation + is not checked at runtime but is only enforced by type checkers. + Usage:: - A generic version of :class:`collections.abc.Reversible`. + class Point2D(TypedDict): + x: int + y: int + label: str -.. class:: SupportsInt + a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK + b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check - An ABC with one abstract method ``__int__``. + assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') -.. class:: SupportsFloat + The type info for introspection can be accessed via ``Point2D.__annotations__`` + and ``Point2D.__total__``. To allow using this feature with older versions + of Python that do not support :pep:`526`, ``TypedDict`` supports two additional + equivalent syntactic forms:: - An ABC with one abstract method ``__float__``. + Point2D = TypedDict('Point2D', x=int, y=int, label=str) + Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) -.. class:: SupportsComplex + By default, all keys must be present in a TypedDict. It is possible + to override this by specifying totality. + Usage:: - An ABC with one abstract method ``__complex__``. + class point2D(TypedDict, total=False): + x: int + y: int -.. class:: SupportsBytes + This means that a point2D TypedDict can have any of the keys omitted. A type + checker is only expected to support a literal False or True as the value of + the total argument. True is the default, and makes all items defined in the + class body be required. - An ABC with one abstract method ``__bytes__``. + See :pep:`589` for more examples and detailed rules of using ``TypedDict``. -.. class:: SupportsIndex + .. versionadded:: 3.8 - An ABC with one abstract method ``__index__``. +.. class:: TypeVar - .. versionadded:: 3.8 + Type variable. -.. class:: SupportsAbs + Usage:: - An ABC with one abstract method ``__abs__`` that is covariant - in its return type. + T = TypeVar('T') # Can be anything + A = TypeVar('A', str, bytes) # Must be str or bytes -.. class:: SupportsRound + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See class Generic for more + information on generic types. Generic functions work as follows:: - An ABC with one abstract method ``__round__`` - that is covariant in its return type. + def repeat(x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n -.. class:: Container(Generic[T_co]) + def longest(x: A, y: A) -> A: + """Return the longest of two strings.""" + return x if len(x) >= len(y) else y - A generic version of :class:`collections.abc.Container`. + The latter example's signature is essentially the overloading + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. -.. class:: Hashable + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issubclass` should not be used with types. - An alias to :class:`collections.abc.Hashable` + Type variables may be marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implicitly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. -.. class:: Sized +.. data:: Union - An alias to :class:`collections.abc.Sized` + Union type; ``Union[X, Y]`` means either X or Y. -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + To define a union, use e.g. ``Union[int, str]``. Details: - A generic version of :class:`collections.abc.Collection` + * The arguments must be types and there must be at least one. - .. versionadded:: 3.6.0 + * Unions of unions are flattened, e.g.:: -.. class:: AbstractSet(Sized, Collection[T_co]) + Union[Union[int, str], float] == Union[int, str, float] - A generic version of :class:`collections.abc.Set`. + * Unions of a single argument vanish, e.g.:: -.. class:: MutableSet(AbstractSet[T]) + Union[int] == int # The constructor actually returns int - A generic version of :class:`collections.abc.MutableSet`. + * Redundant arguments are skipped, e.g.:: -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + Union[int, str, int] == Union[int, str] - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: + * When comparing unions, the argument order is ignored, e.g.:: - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] + Union[int, str] == Union[str, int] -.. class:: MutableMapping(Mapping[KT, VT]) + * You cannot subclass or instantiate a union. - A generic version of :class:`collections.abc.MutableMapping`. + * You cannot write ``Union[X][Y]``. -.. class:: Sequence(Reversible[T_co], Collection[T_co]) + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. - A generic version of :class:`collections.abc.Sequence`. + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. -.. class:: MutableSequence(Sequence[T]) - A generic version of :class:`collections.abc.MutableSequence`. +Protocols +========= -.. class:: ByteString(Sequence[int]) +These types support structural type checking, a.k.a. static duck typing. See :pep:`544`. - A generic version of :class:`collections.abc.ByteString`. - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. +.. class:: Reversible(Iterable[T_co]) - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. + A generic version of :class:`collections.abc.Reversible`. -.. class:: Deque(deque, MutableSequence[T]) +.. class:: SupportsInt - A generic version of :class:`collections.deque`. + An ABC with one abstract method ``__int__``. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 +.. class:: SupportsFloat -.. class:: List(list, MutableSequence[T]) + An ABC with one abstract method ``__float__``. - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. +.. class:: SupportsComplex - This type may be used as follows:: + An ABC with one abstract method ``__complex__``. - T = TypeVar('T', int, float) +.. class:: SupportsBytes - def vec2(x: T, y: T) -> List[T]: - return [x, y] + An ABC with one abstract method ``__bytes__``. - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] +.. class:: SupportsIndex -.. class:: Set(set, MutableSet[T]) + An ABC with one abstract method ``__index__``. - A generic version of :class:`builtins.set `. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. + .. versionadded:: 3.8 -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) +.. class:: SupportsAbs - A generic version of :class:`builtins.frozenset `. + An ABC with one abstract method ``__abs__`` that is covariant + in its return type. -.. class:: MappingView(Sized, Iterable[T_co]) +.. class:: SupportsRound - A generic version of :class:`collections.abc.MappingView`. + An ABC with one abstract method ``__round__`` + that is covariant in its return type. -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) - A generic version of :class:`collections.abc.KeysView`. +Generic ABCs +============ -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) +.. class:: AbstractSet(Sized, Collection[T_co]) - A generic version of :class:`collections.abc.ItemsView`. + A generic version of :class:`collections.abc.Set`. -.. class:: ValuesView(MappingView[VT_co]) +.. class:: AsyncContextManager(Generic[T_co]) - A generic version of :class:`collections.abc.ValuesView`. + A generic version of :class:`contextlib.AbstractAsyncContextManager`. -.. class:: Awaitable(Generic[T_co]) + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - A generic version of :class:`collections.abc.Awaitable`. +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) - .. versionadded:: 3.5.2 + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int + If your generator will only yield values, set the ``SendType`` to + ``None``:: - .. versionadded:: 3.5.3 + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) + + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) + + .. versionadded:: 3.6.1 .. class:: AsyncIterable(Generic[T_co]) @@ -760,56 +990,53 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.5.2 -.. class:: ContextManager(Generic[T_co]) +.. class:: Awaitable(Generic[T_co]) - A generic version of :class:`contextlib.AbstractContextManager`. + A generic version of :class:`collections.abc.Awaitable`. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 + .. versionadded:: 3.5.2 -.. class:: AsyncContextManager(Generic[T_co]) +.. class:: ByteString(Sequence[int]) - A generic version of :class:`contextlib.AbstractAsyncContextManager`. + A generic version of :class:`collections.abc.ByteString`. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. -.. class:: Dict(dict, MutableMapping[KT, VT]) + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - This type can be used as follows:: + A generic version of :class:`collections.abc.Collection` - def count_words(text: str) -> Dict[str, int]: - ... + .. versionadded:: 3.6.0 -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) +.. class:: Container(Generic[T_co]) - A generic version of :class:`collections.defaultdict`. + A generic version of :class:`collections.abc.Container`. - .. versionadded:: 3.5.2 +.. class:: ContextManager(Generic[T_co]) -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) + A generic version of :class:`contextlib.AbstractContextManager`. - A generic version of :class:`collections.OrderedDict`. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 - .. versionadded:: 3.7.2 - -.. class:: Counter(collections.Counter, Dict[T, int]) - - A generic version of :class:`collections.Counter`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: - A generic version of :class:`collections.ChainMap`. + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + .. versionadded:: 3.5.3 .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) @@ -842,71 +1069,143 @@ The module defines the following classes, functions and decorators: yield start start += 1 -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) +.. class:: Hashable - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: + An alias to :class:`collections.abc.Hashable` - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded +.. class:: io.IO + io.TextIO + io.BinaryIO - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. These types are in the ``typing.io`` namespace. - If your generator will only yield values, set the ``SendType`` to - ``None``:: +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) + A generic version of :class:`collections.abc.ItemsView`. - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: +.. class:: Iterable(Generic[T_co]) - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) + A generic version of :class:`collections.abc.Iterable`. - .. versionadded:: 3.6.1 +.. class:: Iterator(Iterable[T_co]) -.. class:: Text + A generic version of :class:`collections.abc.Iterator`. - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: + A generic version of :class:`collections.abc.KeysView`. - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - .. versionadded:: 3.5.2 + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: -.. class:: IO - TextIO - BinaryIO + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. +.. class:: MappingView(Sized, Iterable[T_co]) + + A generic version of :class:`collections.abc.MappingView`. -.. class:: Pattern - Match +.. class:: MutableMapping(Mapping[KT, VT]) + + A generic version of :class:`collections.abc.MutableMapping`. + +.. class:: MutableSequence(Sequence[T]) + + A generic version of :class:`collections.abc.MutableSequence`. + +.. class:: MutableSet(AbstractSet[T]) + + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: re.Pattern + re.Match These type aliases correspond to the return types from :func:`re.compile` and :func:`re.match`. These types (and the corresponding functions) are generic in ``AnyStr`` and can be made specific by writing ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. + ``Match[bytes]``. These types are in the ``typing.re`` namespace. + +.. class:: Sequence(Reversible[T_co], Collection[T_co]) + + A generic version of :class:`collections.abc.Sequence`. + +.. class:: Sized + + An alias to :class:`collections.abc.Sized` + +.. class:: ValuesView(MappingView[VT_co]) + + A generic version of :class:`collections.abc.ValuesView`. + + +Generic Concrete Collections +============================ + +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + + A generic version of :class:`collections.ChainMap`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Counter(collections.Counter, Dict[T, int]) + + A generic version of :class:`collections.Counter`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict`. + + .. versionadded:: 3.5.2 + +.. class:: Deque(deque, MutableSequence[T]) + + A generic version of :class:`collections.deque`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Dict(dict, MutableMapping[KT, VT]) + + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. + + This type can be used as follows:: + + def count_words(text: str) -> Dict[str, int]: + ... + +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) + + A generic version of :class:`builtins.frozenset `. + +.. class:: List(list, MutableSequence[T]) + + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. + + This type may be used as follows:: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] .. class:: NamedTuple @@ -963,74 +1262,36 @@ The module defines the following classes, functions and decorators: The ``_field_types`` and ``__annotations__`` attributes are now regular dictionaries instead of instances of ``OrderedDict``. - .. versionchanged:: 3.9 - Removed the ``_field_types`` attribute in favor of the more - standard ``__annotations__`` attribute which has the same information. - - -.. class:: TypedDict(dict) - - A simple typed namespace. At runtime it is equivalent to - a plain :class:`dict`. - - ``TypedDict`` creates a dictionary type that expects all of its - instances to have a certain set of keys, where each key is - associated with a value of a consistent type. This expectation - is not checked at runtime but is only enforced by type checkers. - Usage:: - - class Point2D(TypedDict): - x: int - y: int - label: str - - a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK - b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check - - assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') - - The type info for introspection can be accessed via ``Point2D.__annotations__`` - and ``Point2D.__total__``. To allow using this feature with older versions - of Python that do not support :pep:`526`, ``TypedDict`` supports two additional - equivalent syntactic forms:: - - Point2D = TypedDict('Point2D', x=int, y=int, label=str) - Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) - By default, all keys must be present in a TypedDict. It is possible - to override this by specifying totality. - Usage:: +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - class point2D(TypedDict, total=False): - x: int - y: int + A generic version of :class:`collections.OrderedDict`. - This means that a point2D TypedDict can have any of the keys omitted. A type - checker is only expected to support a literal False or True as the value of - the total argument. True is the default, and makes all items defined in the - class body be required. + .. versionadded:: 3.7.2 - See :pep:`589` for more examples and detailed rules of using ``TypedDict``. +.. class:: Set(set, MutableSet[T]) - .. versionadded:: 3.8 + A generic version of :class:`builtins.set `. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. -.. class:: ForwardRef +.. data:: Tuple - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. -.. function:: NewType(name, tp) + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - UserId = NewType('UserId', int) - first_user = UserId(1) - .. versionadded:: 3.5.2 +Functions and decorators +======================== .. function:: cast(typ, val) @@ -1041,18 +1302,62 @@ The module defines the following classes, functions and decorators: runtime we intentionally don't check anything (we want this to be as fast as possible). -.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) +.. decorator:: final - Return a dictionary containing type hints for a function, method, module - or class object. + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example:: - This is often the same as ``obj.__annotations__``. In addition, - forward references encoded as string literals are handled by evaluating - them in ``globals`` and ``locals`` namespaces. If necessary, - ``Optional[t]`` is added for function and method annotations if a default - value equal to ``None`` is set. For a class ``C``, return - a dictionary constructed by merging all the ``__annotations__`` along - ``C.__mro__`` in reverse order. + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... + + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 + +.. function:: get_args(tp) +.. function:: get_origin(tp) + + Provide basic introspection for generic types and special typing forms. + + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: + + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) + + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) + + .. versionadded:: 3.8 + +.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) + + Return a dictionary containing type hints for a function, method, module + or class object. + + This is often the same as ``obj.__annotations__``. In addition, + forward references encoded as string literals are handled by evaluating + them in ``globals`` and ``locals`` namespaces. If necessary, + ``Optional[t]`` is added for function and method annotations if a default + value equal to ``None`` is set. For a class ``C``, return + a dictionary constructed by merging all the ``__annotations__`` along + ``C.__mro__`` in reverse order. The function recursively replaces all ``Annotated[T, ...]`` with ``T``, unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for @@ -1070,24 +1375,22 @@ The module defines the following classes, functions and decorators: .. versionchanged:: 3.9 Added ``include_extras`` parameter as part of :pep:`593`. -.. function:: get_origin(tp) -.. function:: get_args(tp) +.. decorator:: no_type_check - Provide basic introspection for generic types and special typing forms. + Decorator to indicate that annotations are not type hints. - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) + This mutates the function(s) in place. - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) +.. decorator:: no_type_check_decorator - .. versionadded:: 3.8 + Decorator to give another decorator the :func:`no_type_check` effect. + + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. .. decorator:: overload @@ -1117,66 +1420,6 @@ The module defines the following classes, functions and decorators: See :pep:`484` for details and comparison with other typing semantics. -.. decorator:: final - - A decorator to indicate to type checkers that the decorated method - cannot be overridden, and the decorated class cannot be subclassed. - For example:: - - class Base: - @final - def done(self) -> None: - ... - class Sub(Base): - def done(self) -> None: # Error reported by type checker - ... - - @final - class Leaf: - ... - class Other(Leaf): # Error reported by type checker - ... - - There is no runtime checking of these properties. See :pep:`591` for - more details. - - .. versionadded:: 3.8 - -.. decorator:: no_type_check - - Decorator to indicate that annotations are not type hints. - - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). - - This mutates the function(s) in place. - -.. decorator:: no_type_check_decorator - - Decorator to give another decorator the :func:`no_type_check` effect. - - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. - -.. decorator:: type_check_only - - Decorator to mark a class or function to be unavailable at runtime. - - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: - - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... - - def fetch_response() -> Response: ... - - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. - .. decorator:: runtime_checkable Mark a protocol class as a runtime protocol. @@ -1197,177 +1440,26 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.8 -.. data:: Any - - Special type indicating an unconstrained type. - - * Every type is compatible with :data:`Any`. - * :data:`Any` is compatible with every type. - -.. data:: NoReturn - - Special type indicating that a function never returns. - For example:: - - from typing import NoReturn - - def stop() -> NoReturn: - raise RuntimeError('no way') - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - -.. data:: Union - - Union type; ``Union[X, Y]`` means either X or Y. - - To define a union, use e.g. ``Union[int, str]``. Details: - - * The arguments must be types and there must be at least one. - - * Unions of unions are flattened, e.g.:: - - Union[Union[int, str], float] == Union[int, str, float] - - * Unions of a single argument vanish, e.g.:: - - Union[int] == int # The constructor actually returns int - - * Redundant arguments are skipped, e.g.:: - - Union[int, str, int] == Union[int, str] - - * When comparing unions, the argument order is ignored, e.g.:: - - Union[int, str] == Union[str, int] - - * You cannot subclass or instantiate a union. - - * You cannot write ``Union[X][Y]``. - - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. - - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. - -.. data:: Optional - - Optional type. - - ``Optional[X]`` is equivalent to ``Union[X, None]``. - - Note that this is not the same concept as an optional argument, - which is one that has a default. An optional argument with a - default does not require the ``Optional`` qualifier on its type - annotation just because it is optional. For example:: - - def foo(arg: int = 0) -> None: - ... - - On the other hand, if an explicit value of ``None`` is allowed, the - use of ``Optional`` is appropriate, whether the argument is optional - or not. For example:: - - def foo(arg: Optional[int] = None) -> None: - ... - -.. data:: Tuple - - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. - - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. - - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - -.. data:: Callable - - Callable type; ``Callable[[int], str]`` is a function of (int) -> str. - - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types or an ellipsis; the return type must be - a single type. - - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. - -.. data:: Literal - - A type that can be used to indicate to type checkers that the - corresponding variable or function parameter has a value equivalent to - the provided literal (or one of several literals). For example:: - - def validate_simple(data: Any) -> Literal[True]: # always returns True - ... - - MODE = Literal['r', 'rb', 'w', 'wb'] - def open_helper(file: str, mode: MODE) -> str: - ... - - open_helper('/some/path', 'r') # Passes type check - open_helper('/other/path', 'typo') # Error in type checker - - ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value - is allowed as type argument to ``Literal[...]``, but type checkers may - impose restrictions. See :pep:`586` for more details about literal types. - - .. versionadded:: 3.8 - -.. data:: ClassVar - - Special type construct to mark class variables. - - As introduced in :pep:`526`, a variable annotation wrapped in ClassVar - indicates that a given attribute is intended to be used as a class variable - and should not be set on instances of that class. Usage:: - - class Starship: - stats: ClassVar[Dict[str, int]] = {} # class variable - damage: int = 10 # instance variable - - :data:`ClassVar` accepts only types and cannot be further subscribed. - - :data:`ClassVar` is not a class itself, and should not - be used with :func:`isinstance` or :func:`issubclass`. - :data:`ClassVar` does not change Python runtime behavior, but - it can be used by third-party type checkers. For example, a type checker - might flag the following code as an error:: - - enterprise_d = Starship(3000) - enterprise_d.stats = {} # Error, setting class variable on instance - Starship.stats = {} # This is OK - - .. versionadded:: 3.5.3 - -.. data:: Final +.. decorator:: type_check_only - A special typing construct to indicate to type checkers that a name - cannot be re-assigned or overridden in a subclass. For example:: + Decorator to mark a class or function to be unavailable at runtime. - MAX_SIZE: Final = 9000 - MAX_SIZE += 1 # Error reported by type checker + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: - class Connection: - TIMEOUT: Final[int] = 10 + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... - class FastConnector(Connection): - TIMEOUT = 1 # Error reported by type checker + def fetch_response() -> Response: ... - There is no runtime checking of these properties. See :pep:`591` for - more details. + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. - .. versionadded:: 3.8 +Aliases and constants +===================== .. data:: AnyStr @@ -1384,6 +1476,20 @@ The module defines the following classes, functions and decorators: concat(b"foo", b"bar") # Ok, output has type 'bytes' concat(u"foo", b"bar") # Error, cannot mix unicode and bytes +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + + .. versionadded:: 3.5.2 + .. data:: TYPE_CHECKING A special constant that is assumed to be ``True`` by 3rd party static @@ -1402,86 +1508,5 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.5.2 -.. data:: Annotated - - A type, introduced in :pep:`593` (``Flexible function and variable - annotations``), to decorate existing types with context-specific metadata - (possibly multiple pieces of it, as ``Annotated`` is variadic). - Specifically, a type ``T`` can be annotated with metadata ``x`` via the - typehint ``Annotated[T, x]``. This metadata can be used for either static - analysis or at runtime. If a library (or tool) encounters a typehint - ``Annotated[T, x]`` and has no special logic for metadata ``x``, it - should ignore it and simply treat the type as ``T``. Unlike the - ``no_type_check`` functionality that currently exists in the ``typing`` - module which completely disables typechecking annotations on a function - or a class, the ``Annotated`` type allows for both static typechecking - of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) - together with runtime access to ``x`` within a specific application. - - Ultimately, the responsibility of how to interpret the annotations (if - at all) is the responsibility of the tool or library encountering the - ``Annotated`` type. A tool or library encountering an ``Annotated`` type - can scan through the annotations to determine if they are of interest - (e.g., using ``isinstance()``). - - When a tool or a library does not support annotations or encounters an - unknown annotation it should just ignore it and treat annotated type as - the underlying type. - It's up to the tool consuming the annotations to decide whether the - client is allowed to have several annotations on one type and how to - merge those annotations. - Since the ``Annotated`` type allows you to put several annotations of - the same (or different) type(s) on any node, the tools or libraries - consuming those annotations are in charge of dealing with potential - duplicates. For example, if you are doing value range analysis you might - allow this:: - - T1 = Annotated[int, ValueRange(-10, 5)] - T2 = Annotated[T1, ValueRange(-20, 3)] - - Passing ``include_extras=True`` to :func:`get_type_hints` lets one - access the extra annotations at runtime. - - The details of the syntax: - - * The first argument to ``Annotated`` must be a valid type - - * Multiple type annotations are supported (``Annotated`` supports variadic - arguments):: - - Annotated[int, ValueRange(3, 10), ctype("char")] - - * ``Annotated`` must be called with at least two arguments ( - ``Annotated[int]`` is not valid) - - * The order of the annotations is preserved and matters for equality - checks:: - - Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ - int, ctype("char"), ValueRange(3, 10) - ] - - * Nested ``Annotated`` types are flattened, with metadata ordered - starting with the innermost annotation:: - - Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ - int, ValueRange(3, 10), ctype("char") - ] - - * Duplicated annotations are not removed:: - - Annotated[int, ValueRange(3, 10)] != Annotated[ - int, ValueRange(3, 10), ValueRange(3, 10) - ] - - * ``Annotated`` can be used with nested and generic aliases:: - - T = TypeVar('T') - Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] - V = Vec[int] - - V == Annotated[List[Tuple[int, int]], MaxLen(10)] - - .. versionadded:: 3.9 From 1ad12ccf3f73d94771bcbb77d924ed658f8c75de Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 10:24:46 -0300 Subject: [PATCH 02/18] bpo-40979: undo changes to go section-by-section --- Doc/library/typing.rst | 1199 ++++++++++++++++++++-------------------- 1 file changed, 587 insertions(+), 612 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d5370988d9c213..e5143c5f8d9e4a 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -435,175 +435,48 @@ can define new custom protocols to fully enjoy structural subtyping (see examples below). -Module contents ---------------- +Classes, functions, and decorators +---------------------------------- The module defines the following classes, functions and decorators: +.. class:: TypeVar -Special typing primitives -========================= - -.. data:: Annotated - - A type, introduced in :pep:`593` (``Flexible function and variable - annotations``), to decorate existing types with context-specific metadata - (possibly multiple pieces of it, as ``Annotated`` is variadic). - Specifically, a type ``T`` can be annotated with metadata ``x`` via the - typehint ``Annotated[T, x]``. This metadata can be used for either static - analysis or at runtime. If a library (or tool) encounters a typehint - ``Annotated[T, x]`` and has no special logic for metadata ``x``, it - should ignore it and simply treat the type as ``T``. Unlike the - ``no_type_check`` functionality that currently exists in the ``typing`` - module which completely disables typechecking annotations on a function - or a class, the ``Annotated`` type allows for both static typechecking - of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) - together with runtime access to ``x`` within a specific application. - - Ultimately, the responsibility of how to interpret the annotations (if - at all) is the responsibility of the tool or library encountering the - ``Annotated`` type. A tool or library encountering an ``Annotated`` type - can scan through the annotations to determine if they are of interest - (e.g., using ``isinstance()``). - - When a tool or a library does not support annotations or encounters an - unknown annotation it should just ignore it and treat annotated type as - the underlying type. - - It's up to the tool consuming the annotations to decide whether the - client is allowed to have several annotations on one type and how to - merge those annotations. - - Since the ``Annotated`` type allows you to put several annotations of - the same (or different) type(s) on any node, the tools or libraries - consuming those annotations are in charge of dealing with potential - duplicates. For example, if you are doing value range analysis you might - allow this:: - - T1 = Annotated[int, ValueRange(-10, 5)] - T2 = Annotated[T1, ValueRange(-20, 3)] - - Passing ``include_extras=True`` to :func:`get_type_hints` lets one - access the extra annotations at runtime. - - The details of the syntax: - - * The first argument to ``Annotated`` must be a valid type - - * Multiple type annotations are supported (``Annotated`` supports variadic - arguments):: - - Annotated[int, ValueRange(3, 10), ctype("char")] - - * ``Annotated`` must be called with at least two arguments ( - ``Annotated[int]`` is not valid) - - * The order of the annotations is preserved and matters for equality - checks:: - - Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ - int, ctype("char"), ValueRange(3, 10) - ] - - * Nested ``Annotated`` types are flattened, with metadata ordered - starting with the innermost annotation:: - - Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ - int, ValueRange(3, 10), ctype("char") - ] - - * Duplicated annotations are not removed:: - - Annotated[int, ValueRange(3, 10)] != Annotated[ - int, ValueRange(3, 10), ValueRange(3, 10) - ] - - * ``Annotated`` can be used with nested and generic aliases:: - - T = TypeVar('T') - Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] - V = Vec[int] - - V == Annotated[List[Tuple[int, int]], MaxLen(10)] - - .. versionadded:: 3.9 - - -.. data:: Any - - Special type indicating an unconstrained type. - - * Every type is compatible with :data:`Any`. - * :data:`Any` is compatible with every type. - -.. data:: Callable - - Callable type; ``Callable[[int], str]`` is a function of (int) -> str. - - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types or an ellipsis; the return type must be - a single type. - - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. - -.. data:: ClassVar - - Special type construct to mark class variables. - - As introduced in :pep:`526`, a variable annotation wrapped in ClassVar - indicates that a given attribute is intended to be used as a class variable - and should not be set on instances of that class. Usage:: - - class Starship: - stats: ClassVar[Dict[str, int]] = {} # class variable - damage: int = 10 # instance variable - - :data:`ClassVar` accepts only types and cannot be further subscribed. - - :data:`ClassVar` is not a class itself, and should not - be used with :func:`isinstance` or :func:`issubclass`. - :data:`ClassVar` does not change Python runtime behavior, but - it can be used by third-party type checkers. For example, a type checker - might flag the following code as an error:: - - enterprise_d = Starship(3000) - enterprise_d.stats = {} # Error, setting class variable on instance - Starship.stats = {} # This is OK - - .. versionadded:: 3.5.3 - -.. data:: Final + Type variable. - A special typing construct to indicate to type checkers that a name - cannot be re-assigned or overridden in a subclass. For example:: + Usage:: - MAX_SIZE: Final = 9000 - MAX_SIZE += 1 # Error reported by type checker + T = TypeVar('T') # Can be anything + A = TypeVar('A', str, bytes) # Must be str or bytes - class Connection: - TIMEOUT: Final[int] = 10 + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See class Generic for more + information on generic types. Generic functions work as follows:: - class FastConnector(Connection): - TIMEOUT = 1 # Error reported by type checker + def repeat(x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n - There is no runtime checking of these properties. See :pep:`591` for - more details. + def longest(x: A, y: A) -> A: + """Return the longest of two strings.""" + return x if len(x) >= len(y) else y - .. versionadded:: 3.8 + The latter example's signature is essentially the overloading + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. -.. class:: ForwardRef + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issubclass` should not be used with types. - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. + Type variables may be marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implicitly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. .. class:: Generic @@ -629,73 +502,6 @@ Special typing primitives except KeyError: return default -.. data:: Literal - - A type that can be used to indicate to type checkers that the - corresponding variable or function parameter has a value equivalent to - the provided literal (or one of several literals). For example:: - - def validate_simple(data: Any) -> Literal[True]: # always returns True - ... - - MODE = Literal['r', 'rb', 'w', 'wb'] - def open_helper(file: str, mode: MODE) -> str: - ... - - open_helper('/some/path', 'r') # Passes type check - open_helper('/other/path', 'typo') # Error in type checker - - ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value - is allowed as type argument to ``Literal[...]``, but type checkers may - impose restrictions. See :pep:`586` for more details about literal types. - - .. versionadded:: 3.8 - -.. function:: NewType(name, tp) - - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: - - UserId = NewType('UserId', int) - first_user = UserId(1) - - .. versionadded:: 3.5.2 - -.. data:: NoReturn - - Special type indicating that a function never returns. - For example:: - - from typing import NoReturn - - def stop() -> NoReturn: - raise RuntimeError('no way') - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - -.. data:: Optional - - Optional type. - - ``Optional[X]`` is equivalent to ``Union[X, None]``. - - Note that this is not the same concept as an optional argument, - which is one that has a default. An optional argument with a - default does not require the ``Optional`` qualifier on its type - annotation just because it is optional. For example:: - - def foo(arg: int = 0) -> None: - ... - - On the other hand, if an explicit value of ``None`` is allowed, the - use of ``Optional`` is appropriate, whether the argument is optional - or not. For example:: - - def foo(arg: Optional[int] = None) -> None: - ... - .. class:: Protocol(Generic) Base class for protocol classes. Protocol classes are defined like this:: @@ -771,212 +577,176 @@ Special typing primitives .. versionadded:: 3.5.2 -.. class:: TypedDict(dict) +.. class:: Iterable(Generic[T_co]) - A simple typed namespace. At runtime it is a plain :class:`dict`. + A generic version of :class:`collections.abc.Iterable`. - ``TypedDict`` creates a dictionary type that expects all of its - instances to have a certain set of keys, where each key is - associated with a value of a consistent type. This expectation - is not checked at runtime but is only enforced by type checkers. - Usage:: - - class Point2D(TypedDict): - x: int - y: int - label: str +.. class:: Iterator(Iterable[T_co]) - a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK - b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check + A generic version of :class:`collections.abc.Iterator`. - assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') +.. class:: Reversible(Iterable[T_co]) - The type info for introspection can be accessed via ``Point2D.__annotations__`` - and ``Point2D.__total__``. To allow using this feature with older versions - of Python that do not support :pep:`526`, ``TypedDict`` supports two additional - equivalent syntactic forms:: + A generic version of :class:`collections.abc.Reversible`. - Point2D = TypedDict('Point2D', x=int, y=int, label=str) - Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) +.. class:: SupportsInt - By default, all keys must be present in a TypedDict. It is possible - to override this by specifying totality. - Usage:: + An ABC with one abstract method ``__int__``. - class point2D(TypedDict, total=False): - x: int - y: int +.. class:: SupportsFloat - This means that a point2D TypedDict can have any of the keys omitted. A type - checker is only expected to support a literal False or True as the value of - the total argument. True is the default, and makes all items defined in the - class body be required. + An ABC with one abstract method ``__float__``. - See :pep:`589` for more examples and detailed rules of using ``TypedDict``. +.. class:: SupportsComplex - .. versionadded:: 3.8 + An ABC with one abstract method ``__complex__``. -.. class:: TypeVar +.. class:: SupportsBytes - Type variable. + An ABC with one abstract method ``__bytes__``. - Usage:: +.. class:: SupportsIndex - T = TypeVar('T') # Can be anything - A = TypeVar('A', str, bytes) # Must be str or bytes + An ABC with one abstract method ``__index__``. - Type variables exist primarily for the benefit of static type - checkers. They serve as the parameters for generic types as well - as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows:: + .. versionadded:: 3.8 - def repeat(x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n +.. class:: SupportsAbs - def longest(x: A, y: A) -> A: - """Return the longest of two strings.""" - return x if len(x) >= len(y) else y + An ABC with one abstract method ``__abs__`` that is covariant + in its return type. - The latter example's signature is essentially the overloading - of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note - that if the arguments are instances of some subclass of :class:`str`, - the return type is still plain :class:`str`. +.. class:: SupportsRound - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. + An ABC with one abstract method ``__round__`` + that is covariant in its return type. - Type variables may be marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. Alternatively, - a type variable may specify an upper bound using ``bound=``. - This means that an actual type substituted (explicitly or implicitly) - for the type variable must be a subclass of the boundary type, - see :pep:`484`. +.. class:: Container(Generic[T_co]) -.. data:: Union + A generic version of :class:`collections.abc.Container`. - Union type; ``Union[X, Y]`` means either X or Y. +.. class:: Hashable - To define a union, use e.g. ``Union[int, str]``. Details: + An alias to :class:`collections.abc.Hashable` - * The arguments must be types and there must be at least one. +.. class:: Sized - * Unions of unions are flattened, e.g.:: + An alias to :class:`collections.abc.Sized` - Union[Union[int, str], float] == Union[int, str, float] +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - * Unions of a single argument vanish, e.g.:: + A generic version of :class:`collections.abc.Collection` - Union[int] == int # The constructor actually returns int + .. versionadded:: 3.6.0 - * Redundant arguments are skipped, e.g.:: +.. class:: AbstractSet(Sized, Collection[T_co]) - Union[int, str, int] == Union[int, str] + A generic version of :class:`collections.abc.Set`. - * When comparing unions, the argument order is ignored, e.g.:: +.. class:: MutableSet(AbstractSet[T]) - Union[int, str] == Union[str, int] + A generic version of :class:`collections.abc.MutableSet`. - * You cannot subclass or instantiate a union. +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - * You cannot write ``Union[X][Y]``. + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. +.. class:: MutableMapping(Mapping[KT, VT]) + A generic version of :class:`collections.abc.MutableMapping`. -Protocols -========= +.. class:: Sequence(Reversible[T_co], Collection[T_co]) -These types support structural type checking, a.k.a. static duck typing. See :pep:`544`. + A generic version of :class:`collections.abc.Sequence`. +.. class:: MutableSequence(Sequence[T]) -.. class:: Reversible(Iterable[T_co]) + A generic version of :class:`collections.abc.MutableSequence`. - A generic version of :class:`collections.abc.Reversible`. +.. class:: ByteString(Sequence[int]) -.. class:: SupportsInt + A generic version of :class:`collections.abc.ByteString`. - An ABC with one abstract method ``__int__``. + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. -.. class:: SupportsFloat + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. - An ABC with one abstract method ``__float__``. +.. class:: Deque(deque, MutableSequence[T]) -.. class:: SupportsComplex + A generic version of :class:`collections.deque`. - An ABC with one abstract method ``__complex__``. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 -.. class:: SupportsBytes +.. class:: List(list, MutableSequence[T]) - An ABC with one abstract method ``__bytes__``. + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. -.. class:: SupportsIndex + This type may be used as follows:: - An ABC with one abstract method ``__index__``. + T = TypeVar('T', int, float) - .. versionadded:: 3.8 + def vec2(x: T, y: T) -> List[T]: + return [x, y] -.. class:: SupportsAbs + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] - An ABC with one abstract method ``__abs__`` that is covariant - in its return type. +.. class:: Set(set, MutableSet[T]) -.. class:: SupportsRound + A generic version of :class:`builtins.set `. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. - An ABC with one abstract method ``__round__`` - that is covariant in its return type. +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) + A generic version of :class:`builtins.frozenset `. -Generic ABCs -============ +.. class:: MappingView(Sized, Iterable[T_co]) -.. class:: AbstractSet(Sized, Collection[T_co]) + A generic version of :class:`collections.abc.MappingView`. - A generic version of :class:`collections.abc.Set`. +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) -.. class:: AsyncContextManager(Generic[T_co]) + A generic version of :class:`collections.abc.KeysView`. - A generic version of :class:`contextlib.AbstractAsyncContextManager`. +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + A generic version of :class:`collections.abc.ItemsView`. -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) +.. class:: ValuesView(MappingView[VT_co]) - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: + A generic version of :class:`collections.abc.ValuesView`. - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded +.. class:: Awaitable(Generic[T_co]) - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. + A generic version of :class:`collections.abc.Awaitable`. - If your generator will only yield values, set the ``SendType`` to - ``None``:: + .. versionadded:: 3.5.2 - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int - .. versionadded:: 3.6.1 + .. versionadded:: 3.5.3 .. class:: AsyncIterable(Generic[T_co]) @@ -990,53 +760,56 @@ Generic ABCs .. versionadded:: 3.5.2 -.. class:: Awaitable(Generic[T_co]) +.. class:: ContextManager(Generic[T_co]) - A generic version of :class:`collections.abc.Awaitable`. + A generic version of :class:`contextlib.AbstractContextManager`. - .. versionadded:: 3.5.2 + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 -.. class:: ByteString(Sequence[int]) +.. class:: AsyncContextManager(Generic[T_co]) - A generic version of :class:`collections.abc.ByteString`. + A generic version of :class:`contextlib.AbstractAsyncContextManager`. - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. +.. class:: Dict(dict, MutableMapping[KT, VT]) -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. - A generic version of :class:`collections.abc.Collection` + This type can be used as follows:: - .. versionadded:: 3.6.0 + def count_words(text: str) -> Dict[str, int]: + ... -.. class:: Container(Generic[T_co]) +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - A generic version of :class:`collections.abc.Container`. + A generic version of :class:`collections.defaultdict`. -.. class:: ContextManager(Generic[T_co]) + .. versionadded:: 3.5.2 - A generic version of :class:`contextlib.AbstractContextManager`. +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 + A generic version of :class:`collections.OrderedDict`. -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + .. versionadded:: 3.7.2 - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: +.. class:: Counter(collections.Counter, Dict[T, int]) - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int + A generic version of :class:`collections.Counter`. - .. versionadded:: 3.5.3 + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + + A generic version of :class:`collections.ChainMap`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) @@ -1069,143 +842,71 @@ Generic ABCs yield start start += 1 -.. class:: Hashable - - An alias to :class:`collections.abc.Hashable` - -.. class:: io.IO - io.TextIO - io.BinaryIO - - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. These types are in the ``typing.io`` namespace. - -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - - A generic version of :class:`collections.abc.ItemsView`. - -.. class:: Iterable(Generic[T_co]) - - A generic version of :class:`collections.abc.Iterable`. +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) -.. class:: Iterator(Iterable[T_co]) + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: - A generic version of :class:`collections.abc.Iterator`. + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. - A generic version of :class:`collections.abc.KeysView`. + If your generator will only yield values, set the ``SendType`` to + ``None``:: -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) -.. class:: MappingView(Sized, Iterable[T_co]) + .. versionadded:: 3.6.1 - A generic version of :class:`collections.abc.MappingView`. +.. class:: Text -.. class:: MutableMapping(Mapping[KT, VT]) + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. - A generic version of :class:`collections.abc.MutableMapping`. + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: -.. class:: MutableSequence(Sequence[T]) + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' - A generic version of :class:`collections.abc.MutableSequence`. + .. versionadded:: 3.5.2 -.. class:: MutableSet(AbstractSet[T]) +.. class:: IO + TextIO + BinaryIO - A generic version of :class:`collections.abc.MutableSet`. + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. -.. class:: re.Pattern - re.Match +.. class:: Pattern + Match These type aliases correspond to the return types from :func:`re.compile` and :func:`re.match`. These types (and the corresponding functions) are generic in ``AnyStr`` and can be made specific by writing ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. These types are in the ``typing.re`` namespace. - -.. class:: Sequence(Reversible[T_co], Collection[T_co]) - - A generic version of :class:`collections.abc.Sequence`. - -.. class:: Sized - - An alias to :class:`collections.abc.Sized` - -.. class:: ValuesView(MappingView[VT_co]) - - A generic version of :class:`collections.abc.ValuesView`. - - -Generic Concrete Collections -============================ - -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) - - A generic version of :class:`collections.ChainMap`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: Counter(collections.Counter, Dict[T, int]) - - A generic version of :class:`collections.Counter`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.defaultdict`. - - .. versionadded:: 3.5.2 - -.. class:: Deque(deque, MutableSequence[T]) - - A generic version of :class:`collections.deque`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: Dict(dict, MutableMapping[KT, VT]) - - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. - - This type can be used as follows:: - - def count_words(text: str) -> Dict[str, int]: - ... - -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - - A generic version of :class:`builtins.frozenset `. - -.. class:: List(list, MutableSequence[T]) - - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. - - This type may be used as follows:: - - T = TypeVar('T', int, float) - - def vec2(x: T, y: T) -> List[T]: - return [x, y] - - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] + ``Match[bytes]``. .. class:: NamedTuple @@ -1262,36 +963,74 @@ Generic Concrete Collections The ``_field_types`` and ``__annotations__`` attributes are now regular dictionaries instead of instances of ``OrderedDict``. + .. versionchanged:: 3.9 + Removed the ``_field_types`` attribute in favor of the more + standard ``__annotations__`` attribute which has the same information. -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - A generic version of :class:`collections.OrderedDict`. +.. class:: TypedDict(dict) - .. versionadded:: 3.7.2 + A simple typed namespace. At runtime it is equivalent to + a plain :class:`dict`. -.. class:: Set(set, MutableSet[T]) + ``TypedDict`` creates a dictionary type that expects all of its + instances to have a certain set of keys, where each key is + associated with a value of a consistent type. This expectation + is not checked at runtime but is only enforced by type checkers. + Usage:: - A generic version of :class:`builtins.set `. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. + class Point2D(TypedDict): + x: int + y: int + label: str -.. data:: Tuple + a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK + b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. + assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. + The type info for introspection can be accessed via ``Point2D.__annotations__`` + and ``Point2D.__total__``. To allow using this feature with older versions + of Python that do not support :pep:`526`, ``TypedDict`` supports two additional + equivalent syntactic forms:: - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + Point2D = TypedDict('Point2D', x=int, y=int, label=str) + Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) + + By default, all keys must be present in a TypedDict. It is possible + to override this by specifying totality. + Usage:: + + class point2D(TypedDict, total=False): + x: int + y: int + + This means that a point2D TypedDict can have any of the keys omitted. A type + checker is only expected to support a literal False or True as the value of + the total argument. True is the default, and makes all items defined in the + class body be required. + + See :pep:`589` for more examples and detailed rules of using ``TypedDict``. + + .. versionadded:: 3.8 + +.. class:: ForwardRef + + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. + +.. function:: NewType(name, tp) + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) -Functions and decorators -======================== + .. versionadded:: 3.5.2 .. function:: cast(typ, val) @@ -1302,62 +1041,18 @@ Functions and decorators runtime we intentionally don't check anything (we want this to be as fast as possible). -.. decorator:: final +.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) - A decorator to indicate to type checkers that the decorated method - cannot be overridden, and the decorated class cannot be subclassed. - For example:: + Return a dictionary containing type hints for a function, method, module + or class object. - class Base: - @final - def done(self) -> None: - ... - class Sub(Base): - def done(self) -> None: # Error reported by type checker - ... - - @final - class Leaf: - ... - class Other(Leaf): # Error reported by type checker - ... - - There is no runtime checking of these properties. See :pep:`591` for - more details. - - .. versionadded:: 3.8 - -.. function:: get_args(tp) -.. function:: get_origin(tp) - - Provide basic introspection for generic types and special typing forms. - - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: - - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) - - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) - - .. versionadded:: 3.8 - -.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) - - Return a dictionary containing type hints for a function, method, module - or class object. - - This is often the same as ``obj.__annotations__``. In addition, - forward references encoded as string literals are handled by evaluating - them in ``globals`` and ``locals`` namespaces. If necessary, - ``Optional[t]`` is added for function and method annotations if a default - value equal to ``None`` is set. For a class ``C``, return - a dictionary constructed by merging all the ``__annotations__`` along - ``C.__mro__`` in reverse order. + This is often the same as ``obj.__annotations__``. In addition, + forward references encoded as string literals are handled by evaluating + them in ``globals`` and ``locals`` namespaces. If necessary, + ``Optional[t]`` is added for function and method annotations if a default + value equal to ``None`` is set. For a class ``C``, return + a dictionary constructed by merging all the ``__annotations__`` along + ``C.__mro__`` in reverse order. The function recursively replaces all ``Annotated[T, ...]`` with ``T``, unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for @@ -1375,22 +1070,24 @@ Functions and decorators .. versionchanged:: 3.9 Added ``include_extras`` parameter as part of :pep:`593`. -.. decorator:: no_type_check - - Decorator to indicate that annotations are not type hints. +.. function:: get_origin(tp) +.. function:: get_args(tp) - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). + Provide basic introspection for generic types and special typing forms. - This mutates the function(s) in place. + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: -.. decorator:: no_type_check_decorator + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) - Decorator to give another decorator the :func:`no_type_check` effect. + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. + .. versionadded:: 3.8 .. decorator:: overload @@ -1420,6 +1117,66 @@ Functions and decorators See :pep:`484` for details and comparison with other typing semantics. +.. decorator:: final + + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example:: + + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... + + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 + +.. decorator:: no_type_check + + Decorator to indicate that annotations are not type hints. + + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). + + This mutates the function(s) in place. + +.. decorator:: no_type_check_decorator + + Decorator to give another decorator the :func:`no_type_check` effect. + + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. + +.. decorator:: type_check_only + + Decorator to mark a class or function to be unavailable at runtime. + + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: + + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... + + def fetch_response() -> Response: ... + + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. + .. decorator:: runtime_checkable Mark a protocol class as a runtime protocol. @@ -1440,26 +1197,177 @@ Functions and decorators .. versionadded:: 3.8 -.. decorator:: type_check_only +.. data:: Any - Decorator to mark a class or function to be unavailable at runtime. + Special type indicating an unconstrained type. - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: + * Every type is compatible with :data:`Any`. + * :data:`Any` is compatible with every type. - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... +.. data:: NoReturn - def fetch_response() -> Response: ... + Special type indicating that a function never returns. + For example:: - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. + from typing import NoReturn -Aliases and constants -===================== + def stop() -> NoReturn: + raise RuntimeError('no way') + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + +.. data:: Union + + Union type; ``Union[X, Y]`` means either X or Y. + + To define a union, use e.g. ``Union[int, str]``. Details: + + * The arguments must be types and there must be at least one. + + * Unions of unions are flattened, e.g.:: + + Union[Union[int, str], float] == Union[int, str, float] + + * Unions of a single argument vanish, e.g.:: + + Union[int] == int # The constructor actually returns int + + * Redundant arguments are skipped, e.g.:: + + Union[int, str, int] == Union[int, str] + + * When comparing unions, the argument order is ignored, e.g.:: + + Union[int, str] == Union[str, int] + + * You cannot subclass or instantiate a union. + + * You cannot write ``Union[X][Y]``. + + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. + +.. data:: Optional + + Optional type. + + ``Optional[X]`` is equivalent to ``Union[X, None]``. + + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ``Optional`` qualifier on its type + annotation just because it is optional. For example:: + + def foo(arg: int = 0) -> None: + ... + + On the other hand, if an explicit value of ``None`` is allowed, the + use of ``Optional`` is appropriate, whether the argument is optional + or not. For example:: + + def foo(arg: Optional[int] = None) -> None: + ... + +.. data:: Tuple + + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. + + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. + + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + +.. data:: Callable + + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. + + The subscription syntax must always be used with exactly two + values: the argument list and the return type. The argument list + must be a list of types or an ellipsis; the return type must be + a single type. + + There is no syntax to indicate optional or keyword arguments; + such function types are rarely used as callback types. + ``Callable[..., ReturnType]`` (literal ellipsis) can be used to + type hint a callable taking any number of arguments and returning + ``ReturnType``. A plain :data:`Callable` is equivalent to + ``Callable[..., Any]``, and in turn to + :class:`collections.abc.Callable`. + +.. data:: Literal + + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent to + the provided literal (or one of several literals). For example:: + + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... + + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... + + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker + + ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value + is allowed as type argument to ``Literal[...]``, but type checkers may + impose restrictions. See :pep:`586` for more details about literal types. + + .. versionadded:: 3.8 + +.. data:: ClassVar + + Special type construct to mark class variables. + + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + :data:`ClassVar` accepts only types and cannot be further subscribed. + + :data:`ClassVar` is not a class itself, and should not + be used with :func:`isinstance` or :func:`issubclass`. + :data:`ClassVar` does not change Python runtime behavior, but + it can be used by third-party type checkers. For example, a type checker + might flag the following code as an error:: + + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK + + .. versionadded:: 3.5.3 + +.. data:: Final + + A special typing construct to indicate to type checkers that a name + cannot be re-assigned or overridden in a subclass. For example:: + + MAX_SIZE: Final = 9000 + MAX_SIZE += 1 # Error reported by type checker + + class Connection: + TIMEOUT: Final[int] = 10 + + class FastConnector(Connection): + TIMEOUT = 1 # Error reported by type checker + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 .. data:: AnyStr @@ -1476,20 +1384,6 @@ Aliases and constants concat(b"foo", b"bar") # Ok, output has type 'bytes' concat(u"foo", b"bar") # Error, cannot mix unicode and bytes -.. class:: Text - - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. - - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: - - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' - - .. versionadded:: 3.5.2 - .. data:: TYPE_CHECKING A special constant that is assumed to be ``True`` by 3rd party static @@ -1508,5 +1402,86 @@ Aliases and constants .. versionadded:: 3.5.2 +.. data:: Annotated + + A type, introduced in :pep:`593` (``Flexible function and variable + annotations``), to decorate existing types with context-specific metadata + (possibly multiple pieces of it, as ``Annotated`` is variadic). + Specifically, a type ``T`` can be annotated with metadata ``x`` via the + typehint ``Annotated[T, x]``. This metadata can be used for either static + analysis or at runtime. If a library (or tool) encounters a typehint + ``Annotated[T, x]`` and has no special logic for metadata ``x``, it + should ignore it and simply treat the type as ``T``. Unlike the + ``no_type_check`` functionality that currently exists in the ``typing`` + module which completely disables typechecking annotations on a function + or a class, the ``Annotated`` type allows for both static typechecking + of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) + together with runtime access to ``x`` within a specific application. + + Ultimately, the responsibility of how to interpret the annotations (if + at all) is the responsibility of the tool or library encountering the + ``Annotated`` type. A tool or library encountering an ``Annotated`` type + can scan through the annotations to determine if they are of interest + (e.g., using ``isinstance()``). + + When a tool or a library does not support annotations or encounters an + unknown annotation it should just ignore it and treat annotated type as + the underlying type. + + It's up to the tool consuming the annotations to decide whether the + client is allowed to have several annotations on one type and how to + merge those annotations. + + Since the ``Annotated`` type allows you to put several annotations of + the same (or different) type(s) on any node, the tools or libraries + consuming those annotations are in charge of dealing with potential + duplicates. For example, if you are doing value range analysis you might + allow this:: + + T1 = Annotated[int, ValueRange(-10, 5)] + T2 = Annotated[T1, ValueRange(-20, 3)] + Passing ``include_extras=True`` to :func:`get_type_hints` lets one + access the extra annotations at runtime. + + The details of the syntax: + * The first argument to ``Annotated`` must be a valid type + + * Multiple type annotations are supported (``Annotated`` supports variadic + arguments):: + + Annotated[int, ValueRange(3, 10), ctype("char")] + + * ``Annotated`` must be called with at least two arguments ( + ``Annotated[int]`` is not valid) + + * The order of the annotations is preserved and matters for equality + checks:: + + Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ + int, ctype("char"), ValueRange(3, 10) + ] + + * Nested ``Annotated`` types are flattened, with metadata ordered + starting with the innermost annotation:: + + Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ + int, ValueRange(3, 10), ctype("char") + ] + + * Duplicated annotations are not removed:: + + Annotated[int, ValueRange(3, 10)] != Annotated[ + int, ValueRange(3, 10), ValueRange(3, 10) + ] + + * ``Annotated`` can be used with nested and generic aliases:: + + T = TypeVar('T') + Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] + V = Vec[int] + + V == Annotated[List[Tuple[int, int]], MaxLen(10)] + + .. versionadded:: 3.9 From c06142bc566f1b73c00b17a3a22f0216fa3628cf Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 10:39:25 -0300 Subject: [PATCH 03/18] bpo-40979: 'Aliases and constants' section --- Doc/library/typing.rst | 105 ++++++++++++++++++++++------------------- 1 file changed, 56 insertions(+), 49 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e5143c5f8d9e4a..14749d89d07f00 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1,3 +1,4 @@ +======================================== :mod:`typing` --- Support for type hints ======================================== @@ -435,11 +436,64 @@ can define new custom protocols to fully enjoy structural subtyping (see examples below). -Classes, functions, and decorators ----------------------------------- +Module contents +--------------- The module defines the following classes, functions and decorators: +Aliases and constants +..................... + +.. data:: AnyStr + + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + +.. data:: TYPE_CHECKING + + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: + + if TYPE_CHECKING: + import expensive_mod + + def fun(arg: 'expensive_mod.SomeType') -> None: + local_var: expensive_mod.AnotherType = other_fun() + + Note that the first type annotation must be enclosed in quotes, making it a + "forward reference", to hide the ``expensive_mod`` reference from the + interpreter runtime. Type annotations for local variables are not + evaluated, so the second annotation does not need to be enclosed in quotes. + + .. versionadded:: 3.5.2 + +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + + .. versionadded:: 3.5.2 + +Remaining classes, functions and decorators +........................................... + .. class:: TypeVar Type variable. @@ -875,20 +929,6 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.6.1 -.. class:: Text - - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. - - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: - - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' - - .. versionadded:: 3.5.2 - .. class:: IO TextIO BinaryIO @@ -1369,39 +1409,6 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.8 -.. data:: AnyStr - - ``AnyStr`` is a type variable defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. - - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: - - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b - - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - -.. data:: TYPE_CHECKING - - A special constant that is assumed to be ``True`` by 3rd party static - type checkers. It is ``False`` at runtime. Usage:: - - if TYPE_CHECKING: - import expensive_mod - - def fun(arg: 'expensive_mod.SomeType') -> None: - local_var: expensive_mod.AnotherType = other_fun() - - Note that the first type annotation must be enclosed in quotes, making it a - "forward reference", to hide the ``expensive_mod`` reference from the - interpreter runtime. Type annotations for local variables are not - evaluated, so the second annotation does not need to be enclosed in quotes. - - .. versionadded:: 3.5.2 - .. data:: Annotated A type, introduced in :pep:`593` (``Flexible function and variable From 7f5d48d803c7b8b37d367f3377116735e0a66a8c Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 10:46:55 -0300 Subject: [PATCH 04/18] bpo-40979: 'Functions and decorators' section --- Doc/library/typing.rst | 333 +++++++++++++++++++++-------------------- 1 file changed, 168 insertions(+), 165 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 14749d89d07f00..b46c202cb137f1 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -491,6 +491,174 @@ Aliases and constants .. versionadded:: 3.5.2 +Functions and decorators +........................ + +.. function:: cast(typ, val) + + Cast a value to a type. + + This returns the value unchanged. To the type checker this + signals that the return value has the designated type, but at + runtime we intentionally don't check anything (we want this + to be as fast as possible). + +.. decorator:: final + + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example:: + + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... + + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 + +.. function:: get_args(tp) +.. function:: get_origin(tp) + + Provide basic introspection for generic types and special typing forms. + + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: + + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) + + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) + + .. versionadded:: 3.8 + +.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) + + Return a dictionary containing type hints for a function, method, module + or class object. + + This is often the same as ``obj.__annotations__``. In addition, + forward references encoded as string literals are handled by evaluating + them in ``globals`` and ``locals`` namespaces. If necessary, + ``Optional[t]`` is added for function and method annotations if a default + value equal to ``None`` is set. For a class ``C``, return + a dictionary constructed by merging all the ``__annotations__`` along + ``C.__mro__`` in reverse order. + + The function recursively replaces all ``Annotated[T, ...]`` with ``T``, + unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for + more information). For example:: + + class Student(NamedTuple): + name: Annotated[str, 'some marker'] + + get_type_hints(Student) == {'name': str} + get_type_hints(Student, include_extras=False) == {'name': str} + get_type_hints(Student, include_extras=True) == { + 'name': Annotated[str, 'some marker'] + } + + .. versionchanged:: 3.9 + Added ``include_extras`` parameter as part of :pep:`593`. + +.. decorator:: no_type_check + + Decorator to indicate that annotations are not type hints. + + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). + + This mutates the function(s) in place. + +.. decorator:: no_type_check_decorator + + Decorator to give another decorator the :func:`no_type_check` effect. + + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. + +.. decorator:: overload + + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + :exc:`NotImplementedError`. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + + + See :pep:`484` for details and comparison with other typing semantics. + +.. decorator:: runtime_checkable + + Mark a protocol class as a runtime protocol. + + Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. + This raises :exc:`TypeError` when applied to a non-protocol class. This + allows a simple-minded structural check, very similar to "one trick ponies" + in :mod:`collections.abc` such as :class:`Iterable`. For example:: + + @runtime_checkable + class Closable(Protocol): + def close(self): ... + + assert isinstance(open('/some/file'), Closable) + + **Warning:** this will check only the presence of the required methods, + not their type signatures! + + .. versionadded:: 3.8 + +.. decorator:: type_check_only + + Decorator to mark a class or function to be unavailable at runtime. + + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: + + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... + + def fetch_response() -> Response: ... + + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. + Remaining classes, functions and decorators ........................................... @@ -1072,171 +1240,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.5.2 -.. function:: cast(typ, val) - - Cast a value to a type. - - This returns the value unchanged. To the type checker this - signals that the return value has the designated type, but at - runtime we intentionally don't check anything (we want this - to be as fast as possible). - -.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) - - Return a dictionary containing type hints for a function, method, module - or class object. - - This is often the same as ``obj.__annotations__``. In addition, - forward references encoded as string literals are handled by evaluating - them in ``globals`` and ``locals`` namespaces. If necessary, - ``Optional[t]`` is added for function and method annotations if a default - value equal to ``None`` is set. For a class ``C``, return - a dictionary constructed by merging all the ``__annotations__`` along - ``C.__mro__`` in reverse order. - - The function recursively replaces all ``Annotated[T, ...]`` with ``T``, - unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for - more information). For example:: - - class Student(NamedTuple): - name: Annotated[str, 'some marker'] - - get_type_hints(Student) == {'name': str} - get_type_hints(Student, include_extras=False) == {'name': str} - get_type_hints(Student, include_extras=True) == { - 'name': Annotated[str, 'some marker'] - } - - .. versionchanged:: 3.9 - Added ``include_extras`` parameter as part of :pep:`593`. - -.. function:: get_origin(tp) -.. function:: get_args(tp) - - Provide basic introspection for generic types and special typing forms. - - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: - - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) - - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) - - .. versionadded:: 3.8 - -.. decorator:: overload - - The ``@overload`` decorator allows describing functions and methods - that support multiple different combinations of argument types. A series - of ``@overload``-decorated definitions must be followed by exactly one - non-``@overload``-decorated definition (for the same function/method). - The ``@overload``-decorated definitions are for the benefit of the - type checker only, since they will be overwritten by the - non-``@overload``-decorated definition, while the latter is used at - runtime but should be ignored by a type checker. At runtime, calling - a ``@overload``-decorated function directly will raise - :exc:`NotImplementedError`. An example of overload that gives a more - precise type than can be expressed using a union or a type variable:: - - @overload - def process(response: None) -> None: - ... - @overload - def process(response: int) -> Tuple[int, str]: - ... - @overload - def process(response: bytes) -> str: - ... - def process(response): - - - See :pep:`484` for details and comparison with other typing semantics. - -.. decorator:: final - - A decorator to indicate to type checkers that the decorated method - cannot be overridden, and the decorated class cannot be subclassed. - For example:: - - class Base: - @final - def done(self) -> None: - ... - class Sub(Base): - def done(self) -> None: # Error reported by type checker - ... - - @final - class Leaf: - ... - class Other(Leaf): # Error reported by type checker - ... - - There is no runtime checking of these properties. See :pep:`591` for - more details. - - .. versionadded:: 3.8 - -.. decorator:: no_type_check - - Decorator to indicate that annotations are not type hints. - - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). - - This mutates the function(s) in place. - -.. decorator:: no_type_check_decorator - - Decorator to give another decorator the :func:`no_type_check` effect. - - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. - -.. decorator:: type_check_only - - Decorator to mark a class or function to be unavailable at runtime. - - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: - - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... - - def fetch_response() -> Response: ... - - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. - -.. decorator:: runtime_checkable - - Mark a protocol class as a runtime protocol. - - Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. - This raises :exc:`TypeError` when applied to a non-protocol class. This - allows a simple-minded structural check, very similar to "one trick ponies" - in :mod:`collections.abc` such as :class:`Iterable`. For example:: - - @runtime_checkable - class Closable(Protocol): - def close(self): ... - - assert isinstance(open('/some/file'), Closable) - - **Warning:** this will check only the presence of the required methods, - not their type signatures! - - .. versionadded:: 3.8 - .. data:: Any Special type indicating an unconstrained type. From 7fb3d72ca53d8dd600f4dbf1f84ef7bc4019cfe8 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 10:57:40 -0300 Subject: [PATCH 05/18] bpo-40979: 'Generic concrete collections' section --- Doc/library/typing.rst | 292 +++++++++++++++++++++-------------------- 1 file changed, 147 insertions(+), 145 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index b46c202cb137f1..f597dbd4bc1077 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -441,6 +441,153 @@ Module contents The module defines the following classes, functions and decorators: +Generic concrete collections +............................ + +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + + A generic version of :class:`collections.ChainMap`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Counter(collections.Counter, Dict[T, int]) + + A generic version of :class:`collections.Counter`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict`. + + .. versionadded:: 3.5.2 + +.. class:: Deque(deque, MutableSequence[T]) + + A generic version of :class:`collections.deque`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Dict(dict, MutableMapping[KT, VT]) + + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. + + This type can be used as follows:: + + def count_words(text: str) -> Dict[str, int]: + ... + +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) + + A generic version of :class:`builtins.frozenset `. + +.. class:: List(list, MutableSequence[T]) + + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. + + This type may be used as follows:: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] + +.. class:: NamedTuple + + Typed version of :func:`collections.namedtuple`. + + Usage:: + + class Employee(NamedTuple): + name: str + id: int + + This is equivalent to:: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + To give a field a default value, you can assign to it in the class body:: + + class Employee(NamedTuple): + name: str + id: int = 3 + + employee = Employee('Guido') + assert employee.id == 3 + + Fields with a default value must come after any fields without a default. + + The resulting class has an extra attribute ``__annotations__`` giving a + dict that maps the field names to the field types. (The field names are in + the ``_fields`` attribute and the default values are in the + ``_field_defaults`` attribute both of which are part of the namedtuple + API.) + + ``NamedTuple`` subclasses can also have docstrings and methods:: + + class Employee(NamedTuple): + """Represents an employee.""" + name: str + id: int = 3 + + def __repr__(self) -> str: + return f'' + + Backward-compatible usage:: + + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) + + .. versionchanged:: 3.6 + Added support for :pep:`526` variable annotation syntax. + + .. versionchanged:: 3.6.1 + Added support for default values, methods, and docstrings. + + .. versionchanged:: 3.8 + The ``_field_types`` and ``__annotations__`` attributes are + now regular dictionaries instead of instances of ``OrderedDict``. + + .. versionchanged:: 3.9 + Removed the ``_field_types`` attribute in favor of the more + standard ``__annotations__`` attribute which has the same information. + +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.OrderedDict`. + + .. versionadded:: 3.7.2 + +.. class:: Set(set, MutableSet[T]) + + A generic version of :class:`builtins.set `. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. + +.. data:: Tuple + + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. + + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. + + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + Aliases and constants ..................... @@ -899,40 +1046,6 @@ Remaining classes, functions and decorators As a shorthand for this type, :class:`bytes` can be used to annotate arguments of any of the types mentioned above. -.. class:: Deque(deque, MutableSequence[T]) - - A generic version of :class:`collections.deque`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: List(list, MutableSequence[T]) - - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. - - This type may be used as follows:: - - T = TypeVar('T', int, float) - - def vec2(x: T, y: T) -> List[T]: - return [x, y] - - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] - -.. class:: Set(set, MutableSet[T]) - - A generic version of :class:`builtins.set `. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. - -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - - A generic version of :class:`builtins.frozenset `. - .. class:: MappingView(Sized, Iterable[T_co]) A generic version of :class:`collections.abc.MappingView`. @@ -996,43 +1109,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.5.4 .. versionadded:: 3.6.2 -.. class:: Dict(dict, MutableMapping[KT, VT]) - - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. - - This type can be used as follows:: - - def count_words(text: str) -> Dict[str, int]: - ... - -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.defaultdict`. - - .. versionadded:: 3.5.2 - -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.OrderedDict`. - - .. versionadded:: 3.7.2 - -.. class:: Counter(collections.Counter, Dict[T, int]) - - A generic version of :class:`collections.Counter`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) - - A generic version of :class:`collections.ChainMap`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -1116,66 +1192,6 @@ Remaining classes, functions and decorators ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or ``Match[bytes]``. -.. class:: NamedTuple - - Typed version of :func:`collections.namedtuple`. - - Usage:: - - class Employee(NamedTuple): - name: str - id: int - - This is equivalent to:: - - Employee = collections.namedtuple('Employee', ['name', 'id']) - - To give a field a default value, you can assign to it in the class body:: - - class Employee(NamedTuple): - name: str - id: int = 3 - - employee = Employee('Guido') - assert employee.id == 3 - - Fields with a default value must come after any fields without a default. - - The resulting class has an extra attribute ``__annotations__`` giving a - dict that maps the field names to the field types. (The field names are in - the ``_fields`` attribute and the default values are in the - ``_field_defaults`` attribute both of which are part of the namedtuple - API.) - - ``NamedTuple`` subclasses can also have docstrings and methods:: - - class Employee(NamedTuple): - """Represents an employee.""" - name: str - id: int = 3 - - def __repr__(self) -> str: - return f'' - - Backward-compatible usage:: - - Employee = NamedTuple('Employee', [('name', str), ('id', int)]) - - .. versionchanged:: 3.6 - Added support for :pep:`526` variable annotation syntax. - - .. versionchanged:: 3.6.1 - Added support for default values, methods, and docstrings. - - .. versionchanged:: 3.8 - The ``_field_types`` and ``__annotations__`` attributes are - now regular dictionaries instead of instances of ``OrderedDict``. - - .. versionchanged:: 3.9 - Removed the ``_field_types`` attribute in favor of the more - standard ``__annotations__`` attribute which has the same information. - - .. class:: TypedDict(dict) A simple typed namespace. At runtime it is equivalent to @@ -1314,20 +1330,6 @@ Remaining classes, functions and decorators def foo(arg: Optional[int] = None) -> None: ... -.. data:: Tuple - - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. - - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. - - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - .. data:: Callable Callable type; ``Callable[[int], str]`` is a function of (int) -> str. From b548112066c250da39b6316dd252e2102f944cde Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 11:05:12 -0300 Subject: [PATCH 06/18] bpo-40979: 'Protocols' section --- Doc/library/typing.rst | 75 ++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 36 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index f597dbd4bc1077..f6f841265f043f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -588,6 +588,45 @@ Generic concrete collections use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. +Protocols +......... + +.. class:: Reversible(Iterable[T_co]) + + A generic version of :class:`collections.abc.Reversible`. + +.. class:: SupportsAbs + + An ABC with one abstract method ``__abs__`` that is covariant + in its return type. + +.. class:: SupportsBytes + + An ABC with one abstract method ``__bytes__``. + +.. class:: SupportsComplex + + An ABC with one abstract method ``__complex__``. + +.. class:: SupportsFloat + + An ABC with one abstract method ``__float__``. + +.. class:: SupportsIndex + + An ABC with one abstract method ``__index__``. + + .. versionadded:: 3.8 + +.. class:: SupportsInt + + An ABC with one abstract method ``__int__``. + +.. class:: SupportsRound + + An ABC with one abstract method ``__round__`` + that is covariant in its return type. + Aliases and constants ..................... @@ -954,42 +993,6 @@ Remaining classes, functions and decorators A generic version of :class:`collections.abc.Iterator`. -.. class:: Reversible(Iterable[T_co]) - - A generic version of :class:`collections.abc.Reversible`. - -.. class:: SupportsInt - - An ABC with one abstract method ``__int__``. - -.. class:: SupportsFloat - - An ABC with one abstract method ``__float__``. - -.. class:: SupportsComplex - - An ABC with one abstract method ``__complex__``. - -.. class:: SupportsBytes - - An ABC with one abstract method ``__bytes__``. - -.. class:: SupportsIndex - - An ABC with one abstract method ``__index__``. - - .. versionadded:: 3.8 - -.. class:: SupportsAbs - - An ABC with one abstract method ``__abs__`` that is covariant - in its return type. - -.. class:: SupportsRound - - An ABC with one abstract method ``__round__`` - that is covariant in its return type. - .. class:: Container(Generic[T_co]) A generic version of :class:`collections.abc.Container`. From d52afa8854982b40dc28e5d228d045614434e365 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 11:22:06 -0300 Subject: [PATCH 07/18] bpo-40979: 'Special typing primitives' section --- Doc/library/typing.rst | 1399 ++++++++++++++++++++-------------------- 1 file changed, 701 insertions(+), 698 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index f6f841265f043f..6205da60db838e 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -441,413 +441,381 @@ Module contents The module defines the following classes, functions and decorators: -Generic concrete collections -............................ - -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) +Special typing primitives +......................... - A generic version of :class:`collections.ChainMap`. +.. data:: Annotated - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + A type, introduced in :pep:`593` (``Flexible function and variable + annotations``), to decorate existing types with context-specific metadata + (possibly multiple pieces of it, as ``Annotated`` is variadic). + Specifically, a type ``T`` can be annotated with metadata ``x`` via the + typehint ``Annotated[T, x]``. This metadata can be used for either static + analysis or at runtime. If a library (or tool) encounters a typehint + ``Annotated[T, x]`` and has no special logic for metadata ``x``, it + should ignore it and simply treat the type as ``T``. Unlike the + ``no_type_check`` functionality that currently exists in the ``typing`` + module which completely disables typechecking annotations on a function + or a class, the ``Annotated`` type allows for both static typechecking + of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) + together with runtime access to ``x`` within a specific application. -.. class:: Counter(collections.Counter, Dict[T, int]) + Ultimately, the responsibility of how to interpret the annotations (if + at all) is the responsibility of the tool or library encountering the + ``Annotated`` type. A tool or library encountering an ``Annotated`` type + can scan through the annotations to determine if they are of interest + (e.g., using ``isinstance()``). - A generic version of :class:`collections.Counter`. + When a tool or a library does not support annotations or encounters an + unknown annotation it should just ignore it and treat annotated type as + the underlying type. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + It's up to the tool consuming the annotations to decide whether the + client is allowed to have several annotations on one type and how to + merge those annotations. -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + Since the ``Annotated`` type allows you to put several annotations of + the same (or different) type(s) on any node, the tools or libraries + consuming those annotations are in charge of dealing with potential + duplicates. For example, if you are doing value range analysis you might + allow this:: - A generic version of :class:`collections.defaultdict`. + T1 = Annotated[int, ValueRange(-10, 5)] + T2 = Annotated[T1, ValueRange(-20, 3)] - .. versionadded:: 3.5.2 + Passing ``include_extras=True`` to :func:`get_type_hints` lets one + access the extra annotations at runtime. -.. class:: Deque(deque, MutableSequence[T]) + The details of the syntax: - A generic version of :class:`collections.deque`. + * The first argument to ``Annotated`` must be a valid type - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + * Multiple type annotations are supported (``Annotated`` supports variadic + arguments):: -.. class:: Dict(dict, MutableMapping[KT, VT]) + Annotated[int, ValueRange(3, 10), ctype("char")] - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. + * ``Annotated`` must be called with at least two arguments ( + ``Annotated[int]`` is not valid) - This type can be used as follows:: + * The order of the annotations is preserved and matters for equality + checks:: - def count_words(text: str) -> Dict[str, int]: - ... + Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ + int, ctype("char"), ValueRange(3, 10) + ] -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) + * Nested ``Annotated`` types are flattened, with metadata ordered + starting with the innermost annotation:: - A generic version of :class:`builtins.frozenset `. + Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ + int, ValueRange(3, 10), ctype("char") + ] -.. class:: List(list, MutableSequence[T]) + * Duplicated annotations are not removed:: - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. + Annotated[int, ValueRange(3, 10)] != Annotated[ + int, ValueRange(3, 10), ValueRange(3, 10) + ] - This type may be used as follows:: + * ``Annotated`` can be used with nested and generic aliases:: - T = TypeVar('T', int, float) + T = TypeVar('T') + Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] + V = Vec[int] - def vec2(x: T, y: T) -> List[T]: - return [x, y] + V == Annotated[List[Tuple[int, int]], MaxLen(10)] - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] + .. versionadded:: 3.9 -.. class:: NamedTuple +.. data:: Any - Typed version of :func:`collections.namedtuple`. + Special type indicating an unconstrained type. - Usage:: + * Every type is compatible with :data:`Any`. + * :data:`Any` is compatible with every type. - class Employee(NamedTuple): - name: str - id: int +.. data:: Callable - This is equivalent to:: + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. - Employee = collections.namedtuple('Employee', ['name', 'id']) + The subscription syntax must always be used with exactly two + values: the argument list and the return type. The argument list + must be a list of types or an ellipsis; the return type must be + a single type. - To give a field a default value, you can assign to it in the class body:: + There is no syntax to indicate optional or keyword arguments; + such function types are rarely used as callback types. + ``Callable[..., ReturnType]`` (literal ellipsis) can be used to + type hint a callable taking any number of arguments and returning + ``ReturnType``. A plain :data:`Callable` is equivalent to + ``Callable[..., Any]``, and in turn to + :class:`collections.abc.Callable`. - class Employee(NamedTuple): - name: str - id: int = 3 +.. data:: ClassVar - employee = Employee('Guido') - assert employee.id == 3 + Special type construct to mark class variables. - Fields with a default value must come after any fields without a default. + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: - The resulting class has an extra attribute ``__annotations__`` giving a - dict that maps the field names to the field types. (The field names are in - the ``_fields`` attribute and the default values are in the - ``_field_defaults`` attribute both of which are part of the namedtuple - API.) + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable - ``NamedTuple`` subclasses can also have docstrings and methods:: + :data:`ClassVar` accepts only types and cannot be further subscribed. - class Employee(NamedTuple): - """Represents an employee.""" - name: str - id: int = 3 + :data:`ClassVar` is not a class itself, and should not + be used with :func:`isinstance` or :func:`issubclass`. + :data:`ClassVar` does not change Python runtime behavior, but + it can be used by third-party type checkers. For example, a type checker + might flag the following code as an error:: - def __repr__(self) -> str: - return f'' + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK - Backward-compatible usage:: + .. versionadded:: 3.5.3 - Employee = NamedTuple('Employee', [('name', str), ('id', int)]) +.. data:: Final - .. versionchanged:: 3.6 - Added support for :pep:`526` variable annotation syntax. + A special typing construct to indicate to type checkers that a name + cannot be re-assigned or overridden in a subclass. For example:: - .. versionchanged:: 3.6.1 - Added support for default values, methods, and docstrings. + MAX_SIZE: Final = 9000 + MAX_SIZE += 1 # Error reported by type checker - .. versionchanged:: 3.8 - The ``_field_types`` and ``__annotations__`` attributes are - now regular dictionaries instead of instances of ``OrderedDict``. + class Connection: + TIMEOUT: Final[int] = 10 - .. versionchanged:: 3.9 - Removed the ``_field_types`` attribute in favor of the more - standard ``__annotations__`` attribute which has the same information. + class FastConnector(Connection): + TIMEOUT = 1 # Error reported by type checker -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) + There is no runtime checking of these properties. See :pep:`591` for + more details. - A generic version of :class:`collections.OrderedDict`. + .. versionadded:: 3.8 - .. versionadded:: 3.7.2 +.. class:: ForwardRef -.. class:: Set(set, MutableSet[T]) + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. - A generic version of :class:`builtins.set `. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. +.. class:: Generic -.. data:: Tuple + Abstract base class for generic types. - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. + A generic type is typically declared by inheriting from an + instantiation of this class with one or more type variables. + For example, a generic mapping type might be defined as:: - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + This class can then be used as follows:: -Protocols -......... + X = TypeVar('X') + Y = TypeVar('Y') -.. class:: Reversible(Iterable[T_co]) + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + try: + return mapping[key] + except KeyError: + return default - A generic version of :class:`collections.abc.Reversible`. +.. data:: Literal -.. class:: SupportsAbs + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent to + the provided literal (or one of several literals). For example:: - An ABC with one abstract method ``__abs__`` that is covariant - in its return type. + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... -.. class:: SupportsBytes + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... - An ABC with one abstract method ``__bytes__``. + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker -.. class:: SupportsComplex + ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value + is allowed as type argument to ``Literal[...]``, but type checkers may + impose restrictions. See :pep:`586` for more details about literal types. - An ABC with one abstract method ``__complex__``. + .. versionadded:: 3.8 -.. class:: SupportsFloat +.. function:: NewType(name, tp) - An ABC with one abstract method ``__float__``. + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: -.. class:: SupportsIndex - - An ABC with one abstract method ``__index__``. - - .. versionadded:: 3.8 - -.. class:: SupportsInt - - An ABC with one abstract method ``__int__``. - -.. class:: SupportsRound - - An ABC with one abstract method ``__round__`` - that is covariant in its return type. - -Aliases and constants -..................... - -.. data:: AnyStr - - ``AnyStr`` is a type variable defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. - - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: - - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b - - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - -.. data:: TYPE_CHECKING - - A special constant that is assumed to be ``True`` by 3rd party static - type checkers. It is ``False`` at runtime. Usage:: + UserId = NewType('UserId', int) + first_user = UserId(1) - if TYPE_CHECKING: - import expensive_mod + .. versionadded:: 3.5.2 - def fun(arg: 'expensive_mod.SomeType') -> None: - local_var: expensive_mod.AnotherType = other_fun() +.. data:: NoReturn - Note that the first type annotation must be enclosed in quotes, making it a - "forward reference", to hide the ``expensive_mod`` reference from the - interpreter runtime. Type annotations for local variables are not - evaluated, so the second annotation does not need to be enclosed in quotes. + Special type indicating that a function never returns. + For example:: - .. versionadded:: 3.5.2 + from typing import NoReturn -.. class:: Text + def stop() -> NoReturn: + raise RuntimeError('no way') - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: +.. data:: Optional - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' + Optional type. - .. versionadded:: 3.5.2 + ``Optional[X]`` is equivalent to ``Union[X, None]``. -Functions and decorators -........................ + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ``Optional`` qualifier on its type + annotation just because it is optional. For example:: -.. function:: cast(typ, val) + def foo(arg: int = 0) -> None: + ... - Cast a value to a type. + On the other hand, if an explicit value of ``None`` is allowed, the + use of ``Optional`` is appropriate, whether the argument is optional + or not. For example:: - This returns the value unchanged. To the type checker this - signals that the return value has the designated type, but at - runtime we intentionally don't check anything (we want this - to be as fast as possible). + def foo(arg: Optional[int] = None) -> None: + ... -.. decorator:: final +.. class:: Protocol(Generic) - A decorator to indicate to type checkers that the decorated method - cannot be overridden, and the decorated class cannot be subclassed. - For example:: + Base class for protocol classes. Protocol classes are defined like this:: - class Base: - @final - def done(self) -> None: + class Proto(Protocol): + def meth(self) -> int: ... - class Sub(Base): - def done(self) -> None: # Error reported by type checker - ... - - @final - class Leaf: - ... - class Other(Leaf): # Error reported by type checker - ... - There is no runtime checking of these properties. See :pep:`591` for - more details. + Such classes are primarily used with static type checkers that recognize + structural subtyping (static duck-typing), for example:: - .. versionadded:: 3.8 + class C: + def meth(self) -> int: + return 0 -.. function:: get_args(tp) -.. function:: get_origin(tp) + def func(x: Proto) -> int: + return x.meth() - Provide basic introspection for generic types and special typing forms. + func(C()) # Passes static type check - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: + See :pep:`544` for details. Protocol classes decorated with + :func:`runtime_checkable` (described later) act as simple-minded runtime + protocols that check only the presence of given attributes, ignoring their + type signatures. - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) + Protocol classes can be generic, for example:: - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) + class GenProto(Protocol[T]): + def meth(self) -> T: + ... .. versionadded:: 3.8 -.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) - - Return a dictionary containing type hints for a function, method, module - or class object. +.. class:: Type(Generic[CT_co]) - This is often the same as ``obj.__annotations__``. In addition, - forward references encoded as string literals are handled by evaluating - them in ``globals`` and ``locals`` namespaces. If necessary, - ``Optional[t]`` is added for function and method annotations if a default - value equal to ``None`` is set. For a class ``C``, return - a dictionary constructed by merging all the ``__annotations__`` along - ``C.__mro__`` in reverse order. + A variable annotated with ``C`` may accept a value of type ``C``. In + contrast, a variable annotated with ``Type[C]`` may accept values that are + classes themselves -- specifically, it will accept the *class object* of + ``C``. For example:: - The function recursively replaces all ``Annotated[T, ...]`` with ``T``, - unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for - more information). For example:: + a = 3 # Has type 'int' + b = int # Has type 'Type[int]' + c = type(a) # Also has type 'Type[int]' - class Student(NamedTuple): - name: Annotated[str, 'some marker'] + Note that ``Type[C]`` is covariant:: - get_type_hints(Student) == {'name': str} - get_type_hints(Student, include_extras=False) == {'name': str} - get_type_hints(Student, include_extras=True) == { - 'name': Annotated[str, 'some marker'] - } + class User: ... + class BasicUser(User): ... + class ProUser(User): ... + class TeamUser(User): ... - .. versionchanged:: 3.9 - Added ``include_extras`` parameter as part of :pep:`593`. + # Accepts User, BasicUser, ProUser, TeamUser, ... + def make_new_user(user_class: Type[User]) -> User: + # ... + return user_class() -.. decorator:: no_type_check + The fact that ``Type[C]`` is covariant implies that all subclasses of + ``C`` should implement the same constructor signature and class method + signatures as ``C``. The type checker should flag violations of this, + but should also allow constructor calls in subclasses that match the + constructor calls in the indicated base class. How the type checker is + required to handle this particular case may change in future revisions of + :pep:`484`. - Decorator to indicate that annotations are not type hints. + The only legal parameters for :class:`Type` are classes, :data:`Any`, + :ref:`type variables `, and unions of any of these types. + For example:: - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). + def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... - This mutates the function(s) in place. + ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent + to ``type``, which is the root of Python's metaclass hierarchy. -.. decorator:: no_type_check_decorator + .. versionadded:: 3.5.2 - Decorator to give another decorator the :func:`no_type_check` effect. +.. class:: TypedDict(dict) - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. + A simple typed namespace. At runtime it is + a plain :class:`dict`. -.. decorator:: overload + ``TypedDict`` creates a dictionary type that expects all of its + instances to have a certain set of keys, where each key is + associated with a value of a consistent type. This expectation + is not checked at runtime but is only enforced by type checkers. + Usage:: - The ``@overload`` decorator allows describing functions and methods - that support multiple different combinations of argument types. A series - of ``@overload``-decorated definitions must be followed by exactly one - non-``@overload``-decorated definition (for the same function/method). - The ``@overload``-decorated definitions are for the benefit of the - type checker only, since they will be overwritten by the - non-``@overload``-decorated definition, while the latter is used at - runtime but should be ignored by a type checker. At runtime, calling - a ``@overload``-decorated function directly will raise - :exc:`NotImplementedError`. An example of overload that gives a more - precise type than can be expressed using a union or a type variable:: + class Point2D(TypedDict): + x: int + y: int + label: str - @overload - def process(response: None) -> None: - ... - @overload - def process(response: int) -> Tuple[int, str]: - ... - @overload - def process(response: bytes) -> str: - ... - def process(response): - + a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK + b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check - See :pep:`484` for details and comparison with other typing semantics. + assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') -.. decorator:: runtime_checkable + The type info for introspection can be accessed via ``Point2D.__annotations__`` + and ``Point2D.__total__``. To allow using this feature with older versions + of Python that do not support :pep:`526`, ``TypedDict`` supports two additional + equivalent syntactic forms:: - Mark a protocol class as a runtime protocol. + Point2D = TypedDict('Point2D', x=int, y=int, label=str) + Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) - Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. - This raises :exc:`TypeError` when applied to a non-protocol class. This - allows a simple-minded structural check, very similar to "one trick ponies" - in :mod:`collections.abc` such as :class:`Iterable`. For example:: + By default, all keys must be present in a TypedDict. It is possible + to override this by specifying totality. + Usage:: - @runtime_checkable - class Closable(Protocol): - def close(self): ... + class point2D(TypedDict, total=False): + x: int + y: int - assert isinstance(open('/some/file'), Closable) + This means that a point2D TypedDict can have any of the keys omitted. A type + checker is only expected to support a literal False or True as the value of + the total argument. True is the default, and makes all items defined in the + class body be required. - **Warning:** this will check only the presence of the required methods, - not their type signatures! + See :pep:`589` for more examples and detailed rules of using ``TypedDict``. .. versionadded:: 3.8 -.. decorator:: type_check_only - - Decorator to mark a class or function to be unavailable at runtime. - - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: - - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... - - def fetch_response() -> Response: ... - - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. - -Remaining classes, functions and decorators -........................................... - .. class:: TypeVar Type variable. @@ -886,617 +854,652 @@ Remaining classes, functions and decorators for the type variable must be a subclass of the boundary type, see :pep:`484`. -.. class:: Generic +.. data:: Union - Abstract base class for generic types. + Union type; ``Union[X, Y]`` means either X or Y. - A generic type is typically declared by inheriting from an - instantiation of this class with one or more type variables. - For example, a generic mapping type might be defined as:: + To define a union, use e.g. ``Union[int, str]``. Details: - class Mapping(Generic[KT, VT]): - def __getitem__(self, key: KT) -> VT: - ... - # Etc. + * The arguments must be types and there must be at least one. - This class can then be used as follows:: + * Unions of unions are flattened, e.g.:: - X = TypeVar('X') - Y = TypeVar('Y') + Union[Union[int, str], float] == Union[int, str, float] - def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: - try: - return mapping[key] - except KeyError: - return default + * Unions of a single argument vanish, e.g.:: -.. class:: Protocol(Generic) + Union[int] == int # The constructor actually returns int - Base class for protocol classes. Protocol classes are defined like this:: + * Redundant arguments are skipped, e.g.:: + + Union[int, str, int] == Union[int, str] + + * When comparing unions, the argument order is ignored, e.g.:: + + Union[int, str] == Union[str, int] + + * You cannot subclass or instantiate a union. + + * You cannot write ``Union[X][Y]``. + + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. + +Generic concrete collections +............................ + +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + + A generic version of :class:`collections.ChainMap`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Counter(collections.Counter, Dict[T, int]) + + A generic version of :class:`collections.Counter`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict`. + + .. versionadded:: 3.5.2 + +.. class:: Deque(deque, MutableSequence[T]) + + A generic version of :class:`collections.deque`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 + +.. class:: Dict(dict, MutableMapping[KT, VT]) + + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. + + This type can be used as follows:: - class Proto(Protocol): - def meth(self) -> int: - ... + def count_words(text: str) -> Dict[str, int]: + ... - Such classes are primarily used with static type checkers that recognize - structural subtyping (static duck-typing), for example:: +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - class C: - def meth(self) -> int: - return 0 + A generic version of :class:`builtins.frozenset `. - def func(x: Proto) -> int: - return x.meth() +.. class:: List(list, MutableSequence[T]) - func(C()) # Passes static type check + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. - See :pep:`544` for details. Protocol classes decorated with - :func:`runtime_checkable` (described later) act as simple-minded runtime - protocols that check only the presence of given attributes, ignoring their - type signatures. + This type may be used as follows:: - Protocol classes can be generic, for example:: + T = TypeVar('T', int, float) - class GenProto(Protocol[T]): - def meth(self) -> T: - ... + def vec2(x: T, y: T) -> List[T]: + return [x, y] - .. versionadded:: 3.8 + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] -.. class:: Type(Generic[CT_co]) +.. class:: NamedTuple - A variable annotated with ``C`` may accept a value of type ``C``. In - contrast, a variable annotated with ``Type[C]`` may accept values that are - classes themselves -- specifically, it will accept the *class object* of - ``C``. For example:: + Typed version of :func:`collections.namedtuple`. - a = 3 # Has type 'int' - b = int # Has type 'Type[int]' - c = type(a) # Also has type 'Type[int]' + Usage:: - Note that ``Type[C]`` is covariant:: + class Employee(NamedTuple): + name: str + id: int - class User: ... - class BasicUser(User): ... - class ProUser(User): ... - class TeamUser(User): ... + This is equivalent to:: - # Accepts User, BasicUser, ProUser, TeamUser, ... - def make_new_user(user_class: Type[User]) -> User: - # ... - return user_class() + Employee = collections.namedtuple('Employee', ['name', 'id']) - The fact that ``Type[C]`` is covariant implies that all subclasses of - ``C`` should implement the same constructor signature and class method - signatures as ``C``. The type checker should flag violations of this, - but should also allow constructor calls in subclasses that match the - constructor calls in the indicated base class. How the type checker is - required to handle this particular case may change in future revisions of - :pep:`484`. + To give a field a default value, you can assign to it in the class body:: - The only legal parameters for :class:`Type` are classes, :data:`Any`, - :ref:`type variables `, and unions of any of these types. - For example:: + class Employee(NamedTuple): + name: str + id: int = 3 - def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... + employee = Employee('Guido') + assert employee.id == 3 - ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent - to ``type``, which is the root of Python's metaclass hierarchy. + Fields with a default value must come after any fields without a default. - .. versionadded:: 3.5.2 + The resulting class has an extra attribute ``__annotations__`` giving a + dict that maps the field names to the field types. (The field names are in + the ``_fields`` attribute and the default values are in the + ``_field_defaults`` attribute both of which are part of the namedtuple + API.) -.. class:: Iterable(Generic[T_co]) + ``NamedTuple`` subclasses can also have docstrings and methods:: - A generic version of :class:`collections.abc.Iterable`. + class Employee(NamedTuple): + """Represents an employee.""" + name: str + id: int = 3 -.. class:: Iterator(Iterable[T_co]) + def __repr__(self) -> str: + return f'' - A generic version of :class:`collections.abc.Iterator`. + Backward-compatible usage:: -.. class:: Container(Generic[T_co]) + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) - A generic version of :class:`collections.abc.Container`. + .. versionchanged:: 3.6 + Added support for :pep:`526` variable annotation syntax. -.. class:: Hashable + .. versionchanged:: 3.6.1 + Added support for default values, methods, and docstrings. - An alias to :class:`collections.abc.Hashable` + .. versionchanged:: 3.8 + The ``_field_types`` and ``__annotations__`` attributes are + now regular dictionaries instead of instances of ``OrderedDict``. -.. class:: Sized + .. versionchanged:: 3.9 + Removed the ``_field_types`` attribute in favor of the more + standard ``__annotations__`` attribute which has the same information. - An alias to :class:`collections.abc.Sized` +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.OrderedDict`. - A generic version of :class:`collections.abc.Collection` + .. versionadded:: 3.7.2 - .. versionadded:: 3.6.0 +.. class:: Set(set, MutableSet[T]) -.. class:: AbstractSet(Sized, Collection[T_co]) + A generic version of :class:`builtins.set `. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. - A generic version of :class:`collections.abc.Set`. +.. data:: Tuple -.. class:: MutableSet(AbstractSet[T]) + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. - A generic version of :class:`collections.abc.MutableSet`. + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: +Protocols +......... - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] +.. class:: Reversible(Iterable[T_co]) -.. class:: MutableMapping(Mapping[KT, VT]) + A generic version of :class:`collections.abc.Reversible`. - A generic version of :class:`collections.abc.MutableMapping`. +.. class:: SupportsAbs -.. class:: Sequence(Reversible[T_co], Collection[T_co]) + An ABC with one abstract method ``__abs__`` that is covariant + in its return type. - A generic version of :class:`collections.abc.Sequence`. +.. class:: SupportsBytes -.. class:: MutableSequence(Sequence[T]) + An ABC with one abstract method ``__bytes__``. - A generic version of :class:`collections.abc.MutableSequence`. +.. class:: SupportsComplex -.. class:: ByteString(Sequence[int]) + An ABC with one abstract method ``__complex__``. - A generic version of :class:`collections.abc.ByteString`. +.. class:: SupportsFloat - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. + An ABC with one abstract method ``__float__``. - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. +.. class:: SupportsIndex -.. class:: MappingView(Sized, Iterable[T_co]) + An ABC with one abstract method ``__index__``. - A generic version of :class:`collections.abc.MappingView`. + .. versionadded:: 3.8 -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) +.. class:: SupportsInt - A generic version of :class:`collections.abc.KeysView`. + An ABC with one abstract method ``__int__``. -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) +.. class:: SupportsRound - A generic version of :class:`collections.abc.ItemsView`. + An ABC with one abstract method ``__round__`` + that is covariant in its return type. -.. class:: ValuesView(MappingView[VT_co]) +Aliases and constants +..................... - A generic version of :class:`collections.abc.ValuesView`. +.. data:: AnyStr -.. class:: Awaitable(Generic[T_co]) + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. - A generic version of :class:`collections.abc.Awaitable`. + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: - .. versionadded:: 3.5.2 + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: +.. data:: TYPE_CHECKING - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: - .. versionadded:: 3.5.3 + if TYPE_CHECKING: + import expensive_mod -.. class:: AsyncIterable(Generic[T_co]) + def fun(arg: 'expensive_mod.SomeType') -> None: + local_var: expensive_mod.AnotherType = other_fun() - A generic version of :class:`collections.abc.AsyncIterable`. + Note that the first type annotation must be enclosed in quotes, making it a + "forward reference", to hide the ``expensive_mod`` reference from the + interpreter runtime. Type annotations for local variables are not + evaluated, so the second annotation does not need to be enclosed in quotes. .. versionadded:: 3.5.2 -.. class:: AsyncIterator(AsyncIterable[T_co]) +.. class:: Text - A generic version of :class:`collections.abc.AsyncIterator`. + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' .. versionadded:: 3.5.2 -.. class:: ContextManager(Generic[T_co]) +Functions and decorators +........................ - A generic version of :class:`contextlib.AbstractContextManager`. +.. function:: cast(typ, val) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 + Cast a value to a type. -.. class:: AsyncContextManager(Generic[T_co]) + This returns the value unchanged. To the type checker this + signals that the return value has the designated type, but at + runtime we intentionally don't check anything (we want this + to be as fast as possible). - A generic version of :class:`contextlib.AbstractAsyncContextManager`. +.. decorator:: final + + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example:: - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... -.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... - A generator can be annotated by the generic type - ``Generator[YieldType, SendType, ReturnType]``. For example:: + There is no runtime checking of these properties. See :pep:`591` for + more details. - def echo_round() -> Generator[int, float, str]: - sent = yield 0 - while sent >= 0: - sent = yield round(sent) - return 'Done' + .. versionadded:: 3.8 - Note that unlike many other generics in the typing module, the ``SendType`` - of :class:`Generator` behaves contravariantly, not covariantly or - invariantly. +.. function:: get_args(tp) +.. function:: get_origin(tp) - If your generator will only yield values, set the ``SendType`` and - ``ReturnType`` to ``None``:: + Provide basic introspection for generic types and special typing forms. - def infinite_stream(start: int) -> Generator[int, None, None]: - while True: - yield start - start += 1 + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: - Alternatively, annotate your generator as having a return type of - either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) - def infinite_stream(start: int) -> Iterator[int]: - while True: - yield start - start += 1 + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) + .. versionadded:: 3.8 - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: +.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded + Return a dictionary containing type hints for a function, method, module + or class object. - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. + This is often the same as ``obj.__annotations__``. In addition, + forward references encoded as string literals are handled by evaluating + them in ``globals`` and ``locals`` namespaces. If necessary, + ``Optional[t]`` is added for function and method annotations if a default + value equal to ``None`` is set. For a class ``C``, return + a dictionary constructed by merging all the ``__annotations__`` along + ``C.__mro__`` in reverse order. - If your generator will only yield values, set the ``SendType`` to - ``None``:: + The function recursively replaces all ``Annotated[T, ...]`` with ``T``, + unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for + more information). For example:: - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) + class Student(NamedTuple): + name: Annotated[str, 'some marker'] - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + get_type_hints(Student) == {'name': str} + get_type_hints(Student, include_extras=False) == {'name': str} + get_type_hints(Student, include_extras=True) == { + 'name': Annotated[str, 'some marker'] + } - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) + .. versionchanged:: 3.9 + Added ``include_extras`` parameter as part of :pep:`593`. - .. versionadded:: 3.6.1 +.. decorator:: no_type_check -.. class:: IO - TextIO - BinaryIO + Decorator to indicate that annotations are not type hints. - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). -.. class:: Pattern - Match + This mutates the function(s) in place. - These type aliases - correspond to the return types from :func:`re.compile` and - :func:`re.match`. These types (and the corresponding functions) - are generic in ``AnyStr`` and can be made specific by writing - ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. +.. decorator:: no_type_check_decorator -.. class:: TypedDict(dict) + Decorator to give another decorator the :func:`no_type_check` effect. - A simple typed namespace. At runtime it is equivalent to - a plain :class:`dict`. + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. - ``TypedDict`` creates a dictionary type that expects all of its - instances to have a certain set of keys, where each key is - associated with a value of a consistent type. This expectation - is not checked at runtime but is only enforced by type checkers. - Usage:: +.. decorator:: overload - class Point2D(TypedDict): - x: int - y: int - label: str + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + :exc:`NotImplementedError`. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: - a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK - b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + - assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') + See :pep:`484` for details and comparison with other typing semantics. - The type info for introspection can be accessed via ``Point2D.__annotations__`` - and ``Point2D.__total__``. To allow using this feature with older versions - of Python that do not support :pep:`526`, ``TypedDict`` supports two additional - equivalent syntactic forms:: +.. decorator:: runtime_checkable - Point2D = TypedDict('Point2D', x=int, y=int, label=str) - Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) + Mark a protocol class as a runtime protocol. - By default, all keys must be present in a TypedDict. It is possible - to override this by specifying totality. - Usage:: + Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. + This raises :exc:`TypeError` when applied to a non-protocol class. This + allows a simple-minded structural check, very similar to "one trick ponies" + in :mod:`collections.abc` such as :class:`Iterable`. For example:: - class point2D(TypedDict, total=False): - x: int - y: int + @runtime_checkable + class Closable(Protocol): + def close(self): ... - This means that a point2D TypedDict can have any of the keys omitted. A type - checker is only expected to support a literal False or True as the value of - the total argument. True is the default, and makes all items defined in the - class body be required. + assert isinstance(open('/some/file'), Closable) - See :pep:`589` for more examples and detailed rules of using ``TypedDict``. + **Warning:** this will check only the presence of the required methods, + not their type signatures! .. versionadded:: 3.8 -.. class:: ForwardRef - - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. +.. decorator:: type_check_only -.. function:: NewType(name, tp) + Decorator to mark a class or function to be unavailable at runtime. - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: - UserId = NewType('UserId', int) - first_user = UserId(1) + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... - .. versionadded:: 3.5.2 + def fetch_response() -> Response: ... -.. data:: Any + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. - Special type indicating an unconstrained type. +Remaining classes, functions and decorators +........................................... - * Every type is compatible with :data:`Any`. - * :data:`Any` is compatible with every type. +.. class:: Iterable(Generic[T_co]) -.. data:: NoReturn + A generic version of :class:`collections.abc.Iterable`. - Special type indicating that a function never returns. - For example:: +.. class:: Iterator(Iterable[T_co]) - from typing import NoReturn + A generic version of :class:`collections.abc.Iterator`. - def stop() -> NoReturn: - raise RuntimeError('no way') +.. class:: Container(Generic[T_co]) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + A generic version of :class:`collections.abc.Container`. -.. data:: Union +.. class:: Hashable - Union type; ``Union[X, Y]`` means either X or Y. + An alias to :class:`collections.abc.Hashable` - To define a union, use e.g. ``Union[int, str]``. Details: +.. class:: Sized - * The arguments must be types and there must be at least one. + An alias to :class:`collections.abc.Sized` - * Unions of unions are flattened, e.g.:: +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - Union[Union[int, str], float] == Union[int, str, float] + A generic version of :class:`collections.abc.Collection` - * Unions of a single argument vanish, e.g.:: + .. versionadded:: 3.6.0 - Union[int] == int # The constructor actually returns int +.. class:: AbstractSet(Sized, Collection[T_co]) - * Redundant arguments are skipped, e.g.:: + A generic version of :class:`collections.abc.Set`. - Union[int, str, int] == Union[int, str] +.. class:: MutableSet(AbstractSet[T]) - * When comparing unions, the argument order is ignored, e.g.:: + A generic version of :class:`collections.abc.MutableSet`. - Union[int, str] == Union[str, int] +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - * You cannot subclass or instantiate a union. + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: - * You cannot write ``Union[X][Y]``. + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. +.. class:: MutableMapping(Mapping[KT, VT]) - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. + A generic version of :class:`collections.abc.MutableMapping`. -.. data:: Optional +.. class:: Sequence(Reversible[T_co], Collection[T_co]) - Optional type. + A generic version of :class:`collections.abc.Sequence`. - ``Optional[X]`` is equivalent to ``Union[X, None]``. +.. class:: MutableSequence(Sequence[T]) - Note that this is not the same concept as an optional argument, - which is one that has a default. An optional argument with a - default does not require the ``Optional`` qualifier on its type - annotation just because it is optional. For example:: + A generic version of :class:`collections.abc.MutableSequence`. - def foo(arg: int = 0) -> None: - ... +.. class:: ByteString(Sequence[int]) - On the other hand, if an explicit value of ``None`` is allowed, the - use of ``Optional`` is appropriate, whether the argument is optional - or not. For example:: + A generic version of :class:`collections.abc.ByteString`. - def foo(arg: Optional[int] = None) -> None: - ... + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. -.. data:: Callable + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. - Callable type; ``Callable[[int], str]`` is a function of (int) -> str. +.. class:: MappingView(Sized, Iterable[T_co]) - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types or an ellipsis; the return type must be - a single type. + A generic version of :class:`collections.abc.MappingView`. - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) -.. data:: Literal + A generic version of :class:`collections.abc.KeysView`. - A type that can be used to indicate to type checkers that the - corresponding variable or function parameter has a value equivalent to - the provided literal (or one of several literals). For example:: +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - def validate_simple(data: Any) -> Literal[True]: # always returns True - ... + A generic version of :class:`collections.abc.ItemsView`. - MODE = Literal['r', 'rb', 'w', 'wb'] - def open_helper(file: str, mode: MODE) -> str: - ... +.. class:: ValuesView(MappingView[VT_co]) - open_helper('/some/path', 'r') # Passes type check - open_helper('/other/path', 'typo') # Error in type checker + A generic version of :class:`collections.abc.ValuesView`. - ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value - is allowed as type argument to ``Literal[...]``, but type checkers may - impose restrictions. See :pep:`586` for more details about literal types. +.. class:: Awaitable(Generic[T_co]) - .. versionadded:: 3.8 + A generic version of :class:`collections.abc.Awaitable`. -.. data:: ClassVar + .. versionadded:: 3.5.2 - Special type construct to mark class variables. +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) - As introduced in :pep:`526`, a variable annotation wrapped in ClassVar - indicates that a given attribute is intended to be used as a class variable - and should not be set on instances of that class. Usage:: + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: - class Starship: - stats: ClassVar[Dict[str, int]] = {} # class variable - damage: int = 10 # instance variable + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int - :data:`ClassVar` accepts only types and cannot be further subscribed. + .. versionadded:: 3.5.3 - :data:`ClassVar` is not a class itself, and should not - be used with :func:`isinstance` or :func:`issubclass`. - :data:`ClassVar` does not change Python runtime behavior, but - it can be used by third-party type checkers. For example, a type checker - might flag the following code as an error:: +.. class:: AsyncIterable(Generic[T_co]) - enterprise_d = Starship(3000) - enterprise_d.stats = {} # Error, setting class variable on instance - Starship.stats = {} # This is OK + A generic version of :class:`collections.abc.AsyncIterable`. - .. versionadded:: 3.5.3 + .. versionadded:: 3.5.2 -.. data:: Final +.. class:: AsyncIterator(AsyncIterable[T_co]) - A special typing construct to indicate to type checkers that a name - cannot be re-assigned or overridden in a subclass. For example:: + A generic version of :class:`collections.abc.AsyncIterator`. - MAX_SIZE: Final = 9000 - MAX_SIZE += 1 # Error reported by type checker + .. versionadded:: 3.5.2 - class Connection: - TIMEOUT: Final[int] = 10 +.. class:: ContextManager(Generic[T_co]) - class FastConnector(Connection): - TIMEOUT = 1 # Error reported by type checker + A generic version of :class:`contextlib.AbstractContextManager`. - There is no runtime checking of these properties. See :pep:`591` for - more details. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 - .. versionadded:: 3.8 +.. class:: AsyncContextManager(Generic[T_co]) -.. data:: Annotated + A generic version of :class:`contextlib.AbstractAsyncContextManager`. - A type, introduced in :pep:`593` (``Flexible function and variable - annotations``), to decorate existing types with context-specific metadata - (possibly multiple pieces of it, as ``Annotated`` is variadic). - Specifically, a type ``T`` can be annotated with metadata ``x`` via the - typehint ``Annotated[T, x]``. This metadata can be used for either static - analysis or at runtime. If a library (or tool) encounters a typehint - ``Annotated[T, x]`` and has no special logic for metadata ``x``, it - should ignore it and simply treat the type as ``T``. Unlike the - ``no_type_check`` functionality that currently exists in the ``typing`` - module which completely disables typechecking annotations on a function - or a class, the ``Annotated`` type allows for both static typechecking - of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) - together with runtime access to ``x`` within a specific application. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - Ultimately, the responsibility of how to interpret the annotations (if - at all) is the responsibility of the tool or library encountering the - ``Annotated`` type. A tool or library encountering an ``Annotated`` type - can scan through the annotations to determine if they are of interest - (e.g., using ``isinstance()``). +.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) - When a tool or a library does not support annotations or encounters an - unknown annotation it should just ignore it and treat annotated type as - the underlying type. + A generator can be annotated by the generic type + ``Generator[YieldType, SendType, ReturnType]``. For example:: - It's up to the tool consuming the annotations to decide whether the - client is allowed to have several annotations on one type and how to - merge those annotations. + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' - Since the ``Annotated`` type allows you to put several annotations of - the same (or different) type(s) on any node, the tools or libraries - consuming those annotations are in charge of dealing with potential - duplicates. For example, if you are doing value range analysis you might - allow this:: + Note that unlike many other generics in the typing module, the ``SendType`` + of :class:`Generator` behaves contravariantly, not covariantly or + invariantly. - T1 = Annotated[int, ValueRange(-10, 5)] - T2 = Annotated[T1, ValueRange(-20, 3)] + If your generator will only yield values, set the ``SendType`` and + ``ReturnType`` to ``None``:: - Passing ``include_extras=True`` to :func:`get_type_hints` lets one - access the extra annotations at runtime. + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 - The details of the syntax: + Alternatively, annotate your generator as having a return type of + either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: - * The first argument to ``Annotated`` must be a valid type + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 - * Multiple type annotations are supported (``Annotated`` supports variadic - arguments):: +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) - Annotated[int, ValueRange(3, 10), ctype("char")] + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: - * ``Annotated`` must be called with at least two arguments ( - ``Annotated[int]`` is not valid) + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded - * The order of the annotations is preserved and matters for equality - checks:: + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. - Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ - int, ctype("char"), ValueRange(3, 10) - ] + If your generator will only yield values, set the ``SendType`` to + ``None``:: - * Nested ``Annotated`` types are flattened, with metadata ordered - starting with the innermost annotation:: + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) - Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ - int, ValueRange(3, 10), ctype("char") - ] + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: - * Duplicated annotations are not removed:: + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) - Annotated[int, ValueRange(3, 10)] != Annotated[ - int, ValueRange(3, 10), ValueRange(3, 10) - ] + .. versionadded:: 3.6.1 - * ``Annotated`` can be used with nested and generic aliases:: +.. class:: IO + TextIO + BinaryIO - T = TypeVar('T') - Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] - V = Vec[int] + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. - V == Annotated[List[Tuple[int, int]], MaxLen(10)] +.. class:: Pattern + Match - .. versionadded:: 3.9 + These type aliases + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. From 49d2a314e69b51a0c0c21875c12d23640dac24bb Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 11:39:16 -0300 Subject: [PATCH 08/18] bpo-40979: 'Generic ABCs' section (initial A) --- Doc/library/typing.rst | 127 +++++++++++++++++++++-------------------- 1 file changed, 65 insertions(+), 62 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 6205da60db838e..0b4e4196a9c6d7 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1291,6 +1291,71 @@ Functions and decorators Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public. +Generic ABCs (with initial A) +............................. + +.. class:: AbstractSet(Sized, Collection[T_co]) + + A generic version of :class:`collections.abc.Set`. + +.. class:: AsyncContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractAsyncContextManager`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) + + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: + + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded + + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. + + If your generator will only yield values, set the ``SendType`` to + ``None``:: + + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) + + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) + + .. versionadded:: 3.6.1 + +.. class:: AsyncIterable(Generic[T_co]) + + A generic version of :class:`collections.abc.AsyncIterable`. + + .. versionadded:: 3.5.2 + +.. class:: AsyncIterator(AsyncIterable[T_co]) + + A generic version of :class:`collections.abc.AsyncIterator`. + + .. versionadded:: 3.5.2 + +.. class:: Awaitable(Generic[T_co]) + + A generic version of :class:`collections.abc.Awaitable`. + + .. versionadded:: 3.5.2 + Remaining classes, functions and decorators ........................................... @@ -1320,10 +1385,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.6.0 -.. class:: AbstractSet(Sized, Collection[T_co]) - - A generic version of :class:`collections.abc.Set`. - .. class:: MutableSet(AbstractSet[T]) A generic version of :class:`collections.abc.MutableSet`. @@ -1374,12 +1435,6 @@ Remaining classes, functions and decorators A generic version of :class:`collections.abc.ValuesView`. -.. class:: Awaitable(Generic[T_co]) - - A generic version of :class:`collections.abc.Awaitable`. - - .. versionadded:: 3.5.2 - .. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) A generic version of :class:`collections.abc.Coroutine`. @@ -1395,18 +1450,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.5.3 -.. class:: AsyncIterable(Generic[T_co]) - - A generic version of :class:`collections.abc.AsyncIterable`. - - .. versionadded:: 3.5.2 - -.. class:: AsyncIterator(AsyncIterable[T_co]) - - A generic version of :class:`collections.abc.AsyncIterator`. - - .. versionadded:: 3.5.2 - .. class:: ContextManager(Generic[T_co]) A generic version of :class:`contextlib.AbstractContextManager`. @@ -1414,13 +1457,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.5.4 .. versionadded:: 3.6.0 -.. class:: AsyncContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractAsyncContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -1452,39 +1488,6 @@ Remaining classes, functions and decorators yield start start += 1 -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) - - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: - - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded - - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. - - If your generator will only yield values, set the ``SendType`` to - ``None``:: - - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) - - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: - - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) - - .. versionadded:: 3.6.1 - .. class:: IO TextIO BinaryIO From bb07243dbbba074c549301223559e33a52290cf4 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 11:45:45 -0300 Subject: [PATCH 09/18] bpo-40979: 'Generic ABCs' section (initials A to H) --- Doc/library/typing.rst | 141 +++++++++++++++++++++-------------------- 1 file changed, 71 insertions(+), 70 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 0b4e4196a9c6d7..d17f04061823a9 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1291,8 +1291,8 @@ Functions and decorators Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public. -Generic ABCs (with initial A) -............................. +Generic ABCs (with initials A to H) +................................... .. class:: AbstractSet(Sized, Collection[T_co]) @@ -1356,59 +1356,6 @@ Generic ABCs (with initial A) .. versionadded:: 3.5.2 -Remaining classes, functions and decorators -........................................... - -.. class:: Iterable(Generic[T_co]) - - A generic version of :class:`collections.abc.Iterable`. - -.. class:: Iterator(Iterable[T_co]) - - A generic version of :class:`collections.abc.Iterator`. - -.. class:: Container(Generic[T_co]) - - A generic version of :class:`collections.abc.Container`. - -.. class:: Hashable - - An alias to :class:`collections.abc.Hashable` - -.. class:: Sized - - An alias to :class:`collections.abc.Sized` - -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - - A generic version of :class:`collections.abc.Collection` - - .. versionadded:: 3.6.0 - -.. class:: MutableSet(AbstractSet[T]) - - A generic version of :class:`collections.abc.MutableSet`. - -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: - - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] - -.. class:: MutableMapping(Mapping[KT, VT]) - - A generic version of :class:`collections.abc.MutableMapping`. - -.. class:: Sequence(Reversible[T_co], Collection[T_co]) - - A generic version of :class:`collections.abc.Sequence`. - -.. class:: MutableSequence(Sequence[T]) - - A generic version of :class:`collections.abc.MutableSequence`. - .. class:: ByteString(Sequence[int]) A generic version of :class:`collections.abc.ByteString`. @@ -1419,21 +1366,22 @@ Remaining classes, functions and decorators As a shorthand for this type, :class:`bytes` can be used to annotate arguments of any of the types mentioned above. -.. class:: MappingView(Sized, Iterable[T_co]) +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - A generic version of :class:`collections.abc.MappingView`. + A generic version of :class:`collections.abc.Collection` -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + .. versionadded:: 3.6.0 - A generic version of :class:`collections.abc.KeysView`. +.. class:: Container(Generic[T_co]) -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.Container`. - A generic version of :class:`collections.abc.ItemsView`. +.. class:: ContextManager(Generic[T_co]) -.. class:: ValuesView(MappingView[VT_co]) + A generic version of :class:`contextlib.AbstractContextManager`. - A generic version of :class:`collections.abc.ValuesView`. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 .. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) @@ -1450,13 +1398,6 @@ Remaining classes, functions and decorators .. versionadded:: 3.5.3 -.. class:: ContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 - .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -1488,6 +1429,66 @@ Remaining classes, functions and decorators yield start start += 1 +.. class:: Hashable + + An alias to :class:`collections.abc.Hashable` + + +Remaining classes, functions and decorators +........................................... + +.. class:: Iterable(Generic[T_co]) + + A generic version of :class:`collections.abc.Iterable`. + +.. class:: Iterator(Iterable[T_co]) + + A generic version of :class:`collections.abc.Iterator`. + +.. class:: Sized + + An alias to :class:`collections.abc.Sized` + +.. class:: MutableSet(AbstractSet[T]) + + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: + + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] + +.. class:: MutableMapping(Mapping[KT, VT]) + + A generic version of :class:`collections.abc.MutableMapping`. + +.. class:: Sequence(Reversible[T_co], Collection[T_co]) + + A generic version of :class:`collections.abc.Sequence`. + +.. class:: MutableSequence(Sequence[T]) + + A generic version of :class:`collections.abc.MutableSequence`. + +.. class:: MappingView(Sized, Iterable[T_co]) + + A generic version of :class:`collections.abc.MappingView`. + +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + + A generic version of :class:`collections.abc.KeysView`. + +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + + A generic version of :class:`collections.abc.ItemsView`. + +.. class:: ValuesView(MappingView[VT_co]) + + A generic version of :class:`collections.abc.ValuesView`. + .. class:: IO TextIO BinaryIO From 5d7c120c5e3f269adb2f1f92f8e7930b9ca74ce6 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 11:51:28 -0300 Subject: [PATCH 10/18] bpo-40979: 'Generic ABCs' section (initials A to K) --- Doc/library/typing.rst | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d17f04061823a9..ce9df7a8cb5c73 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1291,7 +1291,7 @@ Functions and decorators Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public. -Generic ABCs (with initials A to H) +Generic ABCs (with initials A to K) ................................... .. class:: AbstractSet(Sized, Collection[T_co]) @@ -1433,9 +1433,18 @@ Generic ABCs (with initials A to H) An alias to :class:`collections.abc.Hashable` +.. class:: IO + TextIO + BinaryIO -Remaining classes, functions and decorators -........................................... + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. These types are in the `typing.io` namespace. + +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + + A generic version of :class:`collections.abc.ItemsView`. .. class:: Iterable(Generic[T_co]) @@ -1445,6 +1454,13 @@ Remaining classes, functions and decorators A generic version of :class:`collections.abc.Iterator`. +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + + A generic version of :class:`collections.abc.KeysView`. + +Remaining classes, functions and decorators +........................................... + .. class:: Sized An alias to :class:`collections.abc.Sized` @@ -1477,27 +1493,10 @@ Remaining classes, functions and decorators A generic version of :class:`collections.abc.MappingView`. -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) - - A generic version of :class:`collections.abc.KeysView`. - -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - - A generic version of :class:`collections.abc.ItemsView`. - .. class:: ValuesView(MappingView[VT_co]) A generic version of :class:`collections.abc.ValuesView`. -.. class:: IO - TextIO - BinaryIO - - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. - .. class:: Pattern Match From c7358d76453ba3a1cf417077d39eda20c015e534 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 12:01:22 -0300 Subject: [PATCH 11/18] bpo-40979: 'Generic ABCs' section (complete) --- Doc/library/typing.rst | 429 ++++++++++++++++++++--------------------- 1 file changed, 213 insertions(+), 216 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index ce9df7a8cb5c73..aea1fc6f1221ce 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1034,6 +1034,219 @@ Generic concrete collections use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. +Generic ABCs +............ + +.. class:: AbstractSet(Sized, Collection[T_co]) + + A generic version of :class:`collections.abc.Set`. + +.. class:: AsyncContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractAsyncContextManager`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) + + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: + + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded + + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. + + If your generator will only yield values, set the ``SendType`` to + ``None``:: + + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) + + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) + + .. versionadded:: 3.6.1 + +.. class:: AsyncIterable(Generic[T_co]) + + A generic version of :class:`collections.abc.AsyncIterable`. + + .. versionadded:: 3.5.2 + +.. class:: AsyncIterator(AsyncIterable[T_co]) + + A generic version of :class:`collections.abc.AsyncIterator`. + + .. versionadded:: 3.5.2 + +.. class:: Awaitable(Generic[T_co]) + + A generic version of :class:`collections.abc.Awaitable`. + + .. versionadded:: 3.5.2 + +.. class:: ByteString(Sequence[int]) + + A generic version of :class:`collections.abc.ByteString`. + + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. + + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. + +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + + A generic version of :class:`collections.abc.Collection` + + .. versionadded:: 3.6.0 + +.. class:: Container(Generic[T_co]) + + A generic version of :class:`collections.abc.Container`. + +.. class:: ContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractContextManager`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 + +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: + + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int + + .. versionadded:: 3.5.3 + +.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) + + A generator can be annotated by the generic type + ``Generator[YieldType, SendType, ReturnType]``. For example:: + + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' + + Note that unlike many other generics in the typing module, the ``SendType`` + of :class:`Generator` behaves contravariantly, not covariantly or + invariantly. + + If your generator will only yield values, set the ``SendType`` and + ``ReturnType`` to ``None``:: + + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 + + Alternatively, annotate your generator as having a return type of + either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: + + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 + +.. class:: Hashable + + An alias to :class:`collections.abc.Hashable` + +.. class:: IO + TextIO + BinaryIO + + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. These types are in the `typing.io` namespace. + +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + + A generic version of :class:`collections.abc.ItemsView`. + +.. class:: Iterable(Generic[T_co]) + + A generic version of :class:`collections.abc.Iterable`. + +.. class:: Iterator(Iterable[T_co]) + + A generic version of :class:`collections.abc.Iterator`. + +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + + A generic version of :class:`collections.abc.KeysView`. + +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: + + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] + +.. class:: MappingView(Sized, Iterable[T_co]) + + A generic version of :class:`collections.abc.MappingView`. + +.. class:: MutableMapping(Mapping[KT, VT]) + + A generic version of :class:`collections.abc.MutableMapping`. + +.. class:: MutableSequence(Sequence[T]) + + A generic version of :class:`collections.abc.MutableSequence`. + +.. class:: MutableSet(AbstractSet[T]) + + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: Pattern + Match + + These type aliases + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. These types are in the `typing.re` namespace. + +.. class:: Sequence(Reversible[T_co], Collection[T_co]) + + A generic version of :class:`collections.abc.Sequence`. + +.. class:: Sized + + An alias to :class:`collections.abc.Sized` + +.. class:: ValuesView(MappingView[VT_co]) + + A generic version of :class:`collections.abc.ValuesView`. + Protocols ......... @@ -1290,219 +1503,3 @@ Functions and decorators Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public. - -Generic ABCs (with initials A to K) -................................... - -.. class:: AbstractSet(Sized, Collection[T_co]) - - A generic version of :class:`collections.abc.Set`. - -.. class:: AsyncContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractAsyncContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) - - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: - - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded - - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. - - If your generator will only yield values, set the ``SendType`` to - ``None``:: - - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) - - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: - - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) - - .. versionadded:: 3.6.1 - -.. class:: AsyncIterable(Generic[T_co]) - - A generic version of :class:`collections.abc.AsyncIterable`. - - .. versionadded:: 3.5.2 - -.. class:: AsyncIterator(AsyncIterable[T_co]) - - A generic version of :class:`collections.abc.AsyncIterator`. - - .. versionadded:: 3.5.2 - -.. class:: Awaitable(Generic[T_co]) - - A generic version of :class:`collections.abc.Awaitable`. - - .. versionadded:: 3.5.2 - -.. class:: ByteString(Sequence[int]) - - A generic version of :class:`collections.abc.ByteString`. - - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. - - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. - -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - - A generic version of :class:`collections.abc.Collection` - - .. versionadded:: 3.6.0 - -.. class:: Container(Generic[T_co]) - - A generic version of :class:`collections.abc.Container`. - -.. class:: ContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 - -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) - - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: - - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int - - .. versionadded:: 3.5.3 - -.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) - - A generator can be annotated by the generic type - ``Generator[YieldType, SendType, ReturnType]``. For example:: - - def echo_round() -> Generator[int, float, str]: - sent = yield 0 - while sent >= 0: - sent = yield round(sent) - return 'Done' - - Note that unlike many other generics in the typing module, the ``SendType`` - of :class:`Generator` behaves contravariantly, not covariantly or - invariantly. - - If your generator will only yield values, set the ``SendType`` and - ``ReturnType`` to ``None``:: - - def infinite_stream(start: int) -> Generator[int, None, None]: - while True: - yield start - start += 1 - - Alternatively, annotate your generator as having a return type of - either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: - - def infinite_stream(start: int) -> Iterator[int]: - while True: - yield start - start += 1 - -.. class:: Hashable - - An alias to :class:`collections.abc.Hashable` - -.. class:: IO - TextIO - BinaryIO - - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. These types are in the `typing.io` namespace. - -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - - A generic version of :class:`collections.abc.ItemsView`. - -.. class:: Iterable(Generic[T_co]) - - A generic version of :class:`collections.abc.Iterable`. - -.. class:: Iterator(Iterable[T_co]) - - A generic version of :class:`collections.abc.Iterator`. - -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) - - A generic version of :class:`collections.abc.KeysView`. - -Remaining classes, functions and decorators -........................................... - -.. class:: Sized - - An alias to :class:`collections.abc.Sized` - -.. class:: MutableSet(AbstractSet[T]) - - A generic version of :class:`collections.abc.MutableSet`. - -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: - - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] - -.. class:: MutableMapping(Mapping[KT, VT]) - - A generic version of :class:`collections.abc.MutableMapping`. - -.. class:: Sequence(Reversible[T_co], Collection[T_co]) - - A generic version of :class:`collections.abc.Sequence`. - -.. class:: MutableSequence(Sequence[T]) - - A generic version of :class:`collections.abc.MutableSequence`. - -.. class:: MappingView(Sized, Iterable[T_co]) - - A generic version of :class:`collections.abc.MappingView`. - -.. class:: ValuesView(MappingView[VT_co]) - - A generic version of :class:`collections.abc.ValuesView`. - -.. class:: Pattern - Match - - These type aliases - correspond to the return types from :func:`re.compile` and - :func:`re.match`. These types (and the corresponding functions) - are generic in ``AnyStr`` and can be made specific by writing - ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. From 5ab2f2fab4948327bbb08f213828998a614c1fe5 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 12:03:13 -0300 Subject: [PATCH 12/18] bpo-40979: reordering 'Text' entry in 'Aliases and constants' --- Doc/library/typing.rst | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index aea1fc6f1221ce..fe982f0a8f8805 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1304,6 +1304,20 @@ Aliases and constants concat(b"foo", b"bar") # Ok, output has type 'bytes' concat(u"foo", b"bar") # Error, cannot mix unicode and bytes +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + + .. versionadded:: 3.5.2 + .. data:: TYPE_CHECKING A special constant that is assumed to be ``True`` by 3rd party static @@ -1322,20 +1336,6 @@ Aliases and constants .. versionadded:: 3.5.2 -.. class:: Text - - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. - - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: - - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' - - .. versionadded:: 3.5.2 - Functions and decorators ........................ From a7528e3015eae0c76ab772409faa3a9fbbc319d2 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Tue, 21 Jul 2020 12:14:58 -0300 Subject: [PATCH 13/18] bpo-40979: small markup fixes --- Doc/library/typing.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index fe982f0a8f8805..f05ad61d36ec17 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1183,7 +1183,7 @@ Generic ABCs Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` and ``BinaryIO(IO[bytes])`` represent the types of I/O streams such as returned by - :func:`open`. These types are in the `typing.io` namespace. + :func:`open`. These types are in the ``typing.io`` namespace. .. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) @@ -1233,7 +1233,7 @@ Generic ABCs :func:`re.match`. These types (and the corresponding functions) are generic in ``AnyStr`` and can be made specific by writing ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. These types are in the `typing.re` namespace. + ``Match[bytes]``. These types are in the ``typing.re`` namespace. .. class:: Sequence(Reversible[T_co], Collection[T_co]) From 2e25885d243e660a6b5d398b18235ab6173d3af2 Mon Sep 17 00:00:00 2001 From: "blurb-it[bot]" <43283697+blurb-it[bot]@users.noreply.github.com> Date: Tue, 21 Jul 2020 15:23:31 +0000 Subject: [PATCH 14/18] =?UTF-8?q?=F0=9F=93=9C=F0=9F=A4=96=20Added=20by=20b?= =?UTF-8?q?lurb=5Fit.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst | 1 + 1 file changed, 1 insertion(+) create mode 100644 Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst diff --git a/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst b/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst new file mode 100644 index 00000000000000..b0ca4327ad61a4 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst @@ -0,0 +1 @@ +Refactored typing.rst, arranging more than 70 classes, functions, and decorators into new sub-sections. \ No newline at end of file From 4505bbb67274736c79f4dd087f3dc89162f9a469 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Sat, 1 Aug 2020 17:04:00 -0300 Subject: [PATCH 15/18] bpo-40979: sub-sections in Special Types per https://github.com/python/cpython/pull/21574#issuecomment-662214700 --- Doc/library/typing.rst | 377 ++++++++++++++++++++++------------------- 1 file changed, 200 insertions(+), 177 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index f05ad61d36ec17..eaba1ddfda37ed 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -35,7 +35,7 @@ In the function ``greeting``, the argument ``name`` is expected to be of type arguments. Type aliases ------------- +============ A type alias is defined by assigning the type to the alias. In this example, ``Vector`` and ``List[float]`` will be treated as interchangeable synonyms:: @@ -73,7 +73,7 @@ Note that ``None`` as a type hint is a special case and is replaced by .. _distinct: NewType -------- +======= Use the :func:`NewType` helper function to create distinct types:: @@ -150,7 +150,7 @@ See :pep:`484` for more details. .. versionadded:: 3.5.2 Callable --------- +======== Frameworks expecting callback functions of specific signatures might be type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. @@ -173,7 +173,7 @@ for the list of arguments in the type hint: ``Callable[..., ReturnType]``. .. _generics: Generics --------- +======== Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support @@ -200,7 +200,7 @@ called :class:`TypeVar`. User-defined generic types --------------------------- +========================== A user-defined class can be defined as a generic class. @@ -318,7 +318,7 @@ comparable for equality. The :data:`Any` type --------------------- +==================== A special kind of type is :data:`Any`. A static type checker will treat every type as being compatible with :data:`Any` and :data:`Any` as being @@ -396,7 +396,7 @@ manner. Use :data:`Any` to indicate that a value is dynamically typed. Nominal vs structural subtyping -------------------------------- +=============================== Initially :pep:`484` defined Python static type system as using *nominal subtyping*. This means that a class ``A`` is allowed where @@ -435,14 +435,43 @@ Moreover, by subclassing a special class :class:`Protocol`, a user can define new custom protocols to fully enjoy structural subtyping (see examples below). - Module contents ---------------- +=============== The module defines the following classes, functions and decorators: Special typing primitives -......................... +------------------------- + +Special types +""""""""""""" + +These can be used as types in annotations and do not support ``[]``. + +.. data:: Any + + Special type indicating an unconstrained type. + + * Every type is compatible with :data:`Any`. + * :data:`Any` is compatible with every type. + +.. data:: NoReturn + + Special type indicating that a function never returns. + For example:: + + from typing import NoReturn + + def stop() -> NoReturn: + raise RuntimeError('no way') + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + +Special forms +""""""""""""" + +These can be used as types in annotations using ``[]``, each having a unique syntax. .. data:: Annotated @@ -528,13 +557,6 @@ Special typing primitives .. versionadded:: 3.9 -.. data:: Any - - Special type indicating an unconstrained type. - - * Every type is compatible with :data:`Any`. - * :data:`Any` is compatible with every type. - .. data:: Callable Callable type; ``Callable[[int], str]`` is a function of (int) -> str. @@ -597,37 +619,6 @@ Special typing primitives .. versionadded:: 3.8 -.. class:: ForwardRef - - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. - -.. class:: Generic - - Abstract base class for generic types. - - A generic type is typically declared by inheriting from an - instantiation of this class with one or more type variables. - For example, a generic mapping type might be defined as:: - - class Mapping(Generic[KT, VT]): - def __getitem__(self, key: KT) -> VT: - ... - # Etc. - - This class can then be used as follows:: - - X = TypeVar('X') - Y = TypeVar('Y') - - def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: - try: - return mapping[key] - except KeyError: - return default - .. data:: Literal A type that can be used to indicate to type checkers that the @@ -650,30 +641,6 @@ Special typing primitives .. versionadded:: 3.8 -.. function:: NewType(name, tp) - - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: - - UserId = NewType('UserId', int) - first_user = UserId(1) - - .. versionadded:: 3.5.2 - -.. data:: NoReturn - - Special type indicating that a function never returns. - For example:: - - from typing import NoReturn - - def stop() -> NoReturn: - raise RuntimeError('no way') - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - .. data:: Optional Optional type. @@ -695,38 +662,19 @@ Special typing primitives def foo(arg: Optional[int] = None) -> None: ... -.. class:: Protocol(Generic) - - Base class for protocol classes. Protocol classes are defined like this:: - - class Proto(Protocol): - def meth(self) -> int: - ... - - Such classes are primarily used with static type checkers that recognize - structural subtyping (static duck-typing), for example:: - - class C: - def meth(self) -> int: - return 0 - - def func(x: Proto) -> int: - return x.meth() - - func(C()) # Passes static type check - - See :pep:`544` for details. Protocol classes decorated with - :func:`runtime_checkable` (described later) act as simple-minded runtime - protocols that check only the presence of given attributes, ignoring their - type signatures. +.. data:: Tuple - Protocol classes can be generic, for example:: + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. - class GenProto(Protocol[T]): - def meth(self) -> T: - ... + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. - .. versionadded:: 3.8 + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. .. class:: Type(Generic[CT_co]) @@ -770,6 +718,120 @@ Special typing primitives .. versionadded:: 3.5.2 +.. data:: Union + + Union type; ``Union[X, Y]`` means either X or Y. + + To define a union, use e.g. ``Union[int, str]``. Details: + + * The arguments must be types and there must be at least one. + + * Unions of unions are flattened, e.g.:: + + Union[Union[int, str], float] == Union[int, str, float] + + * Unions of a single argument vanish, e.g.:: + + Union[int] == int # The constructor actually returns int + + * Redundant arguments are skipped, e.g.:: + + Union[int, str, int] == Union[int, str] + + * When comparing unions, the argument order is ignored, e.g.:: + + Union[int, str] == Union[str, int] + + * You cannot subclass or instantiate a union. + + * You cannot write ``Union[X][Y]``. + + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. + +Other special directives +"""""""""""""""""""""""" + +These are not used in annotations. They are building blocks used for declaring types. + + +.. class:: ForwardRef + + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. + +.. class:: Generic + + Abstract base class for generic types. + + A generic type is typically declared by inheriting from an + instantiation of this class with one or more type variables. + For example, a generic mapping type might be defined as:: + + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. + + This class can then be used as follows:: + + X = TypeVar('X') + Y = TypeVar('Y') + + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + try: + return mapping[key] + except KeyError: + return default + +.. function:: NewType(name, tp) + + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) + + .. versionadded:: 3.5.2 + +.. class:: Protocol(Generic) + + Base class for protocol classes. Protocol classes are defined like this:: + + class Proto(Protocol): + def meth(self) -> int: + ... + + Such classes are primarily used with static type checkers that recognize + structural subtyping (static duck-typing), for example:: + + class C: + def meth(self) -> int: + return 0 + + def func(x: Proto) -> int: + return x.meth() + + func(C()) # Passes static type check + + See :pep:`544` for details. Protocol classes decorated with + :func:`runtime_checkable` (described later) act as simple-minded runtime + protocols that check only the presence of given attributes, ignoring their + type signatures. + + Protocol classes can be generic, for example:: + + class GenProto(Protocol[T]): + def meth(self) -> T: + ... + + .. versionadded:: 3.8 + .. class:: TypedDict(dict) A simple typed namespace. At runtime it is @@ -854,41 +916,54 @@ Special typing primitives for the type variable must be a subclass of the boundary type, see :pep:`484`. -.. data:: Union +Generic concrete collections +"""""""""""""""""""""""""""" - Union type; ``Union[X, Y]`` means either X or Y. +Corresponding to built-in types +............................... - To define a union, use e.g. ``Union[int, str]``. Details: +.. class:: Dict(dict, MutableMapping[KT, VT]) - * The arguments must be types and there must be at least one. + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. - * Unions of unions are flattened, e.g.:: + This type can be used as follows:: - Union[Union[int, str], float] == Union[int, str, float] + def count_words(text: str) -> Dict[str, int]: + ... - * Unions of a single argument vanish, e.g.:: +.. class:: List(list, MutableSequence[T]) - Union[int] == int # The constructor actually returns int + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. - * Redundant arguments are skipped, e.g.:: + This type may be used as follows:: - Union[int, str, int] == Union[int, str] + T = TypeVar('T', int, float) - * When comparing unions, the argument order is ignored, e.g.:: + def vec2(x: T, y: T) -> List[T]: + return [x, y] - Union[int, str] == Union[str, int] + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] - * You cannot subclass or instantiate a union. +.. class:: Set(set, MutableSet[T]) - * You cannot write ``Union[X][Y]``. + A generic version of :class:`builtins.set `. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. + A generic version of :class:`builtins.frozenset `. -Generic concrete collections -............................ +.. note:: :data:`Tuple` is a special form. + +Corresponding to types in :mod:`collections` +............................................ .. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) @@ -917,38 +992,6 @@ Generic concrete collections .. versionadded:: 3.5.4 .. versionadded:: 3.6.1 -.. class:: Dict(dict, MutableMapping[KT, VT]) - - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. - - This type can be used as follows:: - - def count_words(text: str) -> Dict[str, int]: - ... - -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - - A generic version of :class:`builtins.frozenset `. - -.. class:: List(list, MutableSequence[T]) - - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. - - This type may be used as follows:: - - T = TypeVar('T', int, float) - - def vec2(x: T, y: T) -> List[T]: - return [x, y] - - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] - .. class:: NamedTuple Typed version of :func:`collections.namedtuple`. @@ -1014,28 +1057,8 @@ Generic concrete collections .. versionadded:: 3.7.2 -.. class:: Set(set, MutableSet[T]) - - A generic version of :class:`builtins.set `. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. - -.. data:: Tuple - - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. - - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. - - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - -Generic ABCs -............ +Other ABCs +"""""""""" .. class:: AbstractSet(Sized, Collection[T_co]) @@ -1248,7 +1271,7 @@ Generic ABCs A generic version of :class:`collections.abc.ValuesView`. Protocols -......... +""""""""" .. class:: Reversible(Iterable[T_co]) @@ -1287,7 +1310,7 @@ Protocols that is covariant in its return type. Aliases and constants -..................... +""""""""""""""""""""" .. data:: AnyStr @@ -1337,7 +1360,7 @@ Aliases and constants .. versionadded:: 3.5.2 Functions and decorators -........................ +"""""""""""""""""""""""" .. function:: cast(typ, val) From cd6e8eab66814f3946c894feffcfbd960a868016 Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Sat, 1 Aug 2020 21:29:14 -0300 Subject: [PATCH 16/18] bpo-40979: further reordering after first review --- Doc/library/typing.rst | 371 ++++++++++++++++++++++------------------- 1 file changed, 198 insertions(+), 173 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index eaba1ddfda37ed..6bbcb9fd6de384 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -756,7 +756,6 @@ Other special directives These are not used in annotations. They are building blocks used for declaring types. - .. class:: ForwardRef A class used for internal typing representation of string forward references. @@ -916,11 +915,29 @@ These are not used in annotations. They are building blocks used for declaring t for the type variable must be a subclass of the boundary type, see :pep:`484`. +Predefined type variable +"""""""""""""""""""""""" + +.. data:: AnyStr + + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + Generic concrete collections -"""""""""""""""""""""""""""" +---------------------------- Corresponding to built-in types -............................... +""""""""""""""""""""""""""""""" .. class:: Dict(dict, MutableMapping[KT, VT]) @@ -963,7 +980,19 @@ Corresponding to built-in types .. note:: :data:`Tuple` is a special form. Corresponding to types in :mod:`collections` -............................................ +"""""""""""""""""""""""""""""""""""""""""""" + +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict`. + + .. versionadded:: 3.5.2 + +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.OrderedDict`. + + .. versionadded:: 3.7.2 .. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) @@ -979,12 +1008,6 @@ Corresponding to types in :mod:`collections` .. versionadded:: 3.5.4 .. versionadded:: 3.6.1 -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.defaultdict`. - - .. versionadded:: 3.5.2 - .. class:: Deque(deque, MutableSequence[T]) A generic version of :class:`collections.deque`. @@ -1051,96 +1074,116 @@ Corresponding to types in :mod:`collections` Removed the ``_field_types`` attribute in favor of the more standard ``__annotations__`` attribute which has the same information. -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - A generic version of :class:`collections.OrderedDict`. +Other concrete types +"""""""""""""""""""" - .. versionadded:: 3.7.2 +.. class:: IO + TextIO + BinaryIO -Other ABCs -"""""""""" + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. These types are in the ``typing.io`` namespace. + +.. class:: Pattern + Match + + These type aliases + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. These types are in the ``typing.re`` namespace. + +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + + .. versionadded:: 3.5.2 + +Abstract Base Classes +--------------------- + +Corresponding to collections in :mod:`collections.abc` +"""""""""""""""""""""""""""""""""""""""""""""""""""""" .. class:: AbstractSet(Sized, Collection[T_co]) A generic version of :class:`collections.abc.Set`. -.. class:: AsyncContextManager(Generic[T_co]) +.. class:: ByteString(Sequence[int]) - A generic version of :class:`contextlib.AbstractAsyncContextManager`. + A generic version of :class:`collections.abc.ByteString`. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded + A generic version of :class:`collections.abc.Collection` - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. + .. versionadded:: 3.6.0 - If your generator will only yield values, set the ``SendType`` to - ``None``:: +.. class:: Container(Generic[T_co]) - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) + A generic version of :class:`collections.abc.Container`. - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) + A generic version of :class:`collections.abc.ItemsView`. - .. versionadded:: 3.6.1 +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) -.. class:: AsyncIterable(Generic[T_co]) + A generic version of :class:`collections.abc.KeysView`. - A generic version of :class:`collections.abc.AsyncIterable`. +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - .. versionadded:: 3.5.2 + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: -.. class:: AsyncIterator(AsyncIterable[T_co]) + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] - A generic version of :class:`collections.abc.AsyncIterator`. +.. class:: MappingView(Sized, Iterable[T_co]) - .. versionadded:: 3.5.2 + A generic version of :class:`collections.abc.MappingView`. -.. class:: Awaitable(Generic[T_co]) +.. class:: MutableMapping(Mapping[KT, VT]) - A generic version of :class:`collections.abc.Awaitable`. + A generic version of :class:`collections.abc.MutableMapping`. - .. versionadded:: 3.5.2 +.. class:: MutableSequence(Sequence[T]) -.. class:: ByteString(Sequence[int]) + A generic version of :class:`collections.abc.MutableSequence`. - A generic version of :class:`collections.abc.ByteString`. +.. class:: MutableSet(AbstractSet[T]) - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. + A generic version of :class:`collections.abc.MutableSet`. - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. +.. class:: Sequence(Reversible[T_co], Collection[T_co]) -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Sequence`. - A generic version of :class:`collections.abc.Collection` +.. class:: ValuesView(MappingView[VT_co]) - .. versionadded:: 3.6.0 + A generic version of :class:`collections.abc.ValuesView`. -.. class:: Container(Generic[T_co]) - A generic version of :class:`collections.abc.Container`. +Corresponding to other types in :mod:`collections.abc` +"""""""""""""""""""""""""""""""""""""""""""""""""""""" .. class:: ContextManager(Generic[T_co]) @@ -1149,20 +1192,13 @@ Other ABCs .. versionadded:: 3.5.4 .. versionadded:: 3.6.0 -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) +.. class:: Iterable(Generic[T_co]) - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: + A generic version of :class:`collections.abc.Iterable`. - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int +.. class:: Iterator(Iterable[T_co]) - .. versionadded:: 3.5.3 + A generic version of :class:`collections.abc.Iterator`. .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) @@ -1199,83 +1235,94 @@ Other ABCs An alias to :class:`collections.abc.Hashable` -.. class:: IO - TextIO - BinaryIO +.. class:: Reversible(Iterable[T_co]) - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. These types are in the ``typing.io`` namespace. + A generic version of :class:`collections.abc.Reversible`. -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) +.. class:: Sized - A generic version of :class:`collections.abc.ItemsView`. + An alias to :class:`collections.abc.Sized` -.. class:: Iterable(Generic[T_co]) +Asynchronous programming +"""""""""""""""""""""""" - A generic version of :class:`collections.abc.Iterable`. +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) -.. class:: Iterator(Iterable[T_co]) + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: - A generic version of :class:`collections.abc.Iterator`. + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + .. versionadded:: 3.5.3 - A generic version of :class:`collections.abc.KeysView`. +.. class:: AsyncContextManager(Generic[T_co]) -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + A generic version of :class:`contextlib.AbstractAsyncContextManager`. - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) -.. class:: MappingView(Sized, Iterable[T_co]) + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: - A generic version of :class:`collections.abc.MappingView`. + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded -.. class:: MutableMapping(Mapping[KT, VT]) + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. - A generic version of :class:`collections.abc.MutableMapping`. + If your generator will only yield values, set the ``SendType`` to + ``None``:: -.. class:: MutableSequence(Sequence[T]) + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) - A generic version of :class:`collections.abc.MutableSequence`. + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: -.. class:: MutableSet(AbstractSet[T]) + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) - A generic version of :class:`collections.abc.MutableSet`. + .. versionadded:: 3.6.1 -.. class:: Pattern - Match +.. class:: AsyncIterable(Generic[T_co]) - These type aliases - correspond to the return types from :func:`re.compile` and - :func:`re.match`. These types (and the corresponding functions) - are generic in ``AnyStr`` and can be made specific by writing - ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. These types are in the ``typing.re`` namespace. + A generic version of :class:`collections.abc.AsyncIterable`. -.. class:: Sequence(Reversible[T_co], Collection[T_co]) + .. versionadded:: 3.5.2 - A generic version of :class:`collections.abc.Sequence`. +.. class:: AsyncIterator(AsyncIterable[T_co]) -.. class:: Sized + A generic version of :class:`collections.abc.AsyncIterator`. - An alias to :class:`collections.abc.Sized` + .. versionadded:: 3.5.2 -.. class:: ValuesView(MappingView[VT_co]) +.. class:: Awaitable(Generic[T_co]) - A generic version of :class:`collections.abc.ValuesView`. + A generic version of :class:`collections.abc.Awaitable`. -Protocols -""""""""" + .. versionadded:: 3.5.2 -.. class:: Reversible(Iterable[T_co]) +Protocols +--------- - A generic version of :class:`collections.abc.Reversible`. +These protocols are decorated with :func:`runtime_checkable`. .. class:: SupportsAbs @@ -1309,58 +1356,8 @@ Protocols An ABC with one abstract method ``__round__`` that is covariant in its return type. -Aliases and constants -""""""""""""""""""""" - -.. data:: AnyStr - - ``AnyStr`` is a type variable defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. - - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: - - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b - - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - -.. class:: Text - - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. - - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: - - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' - - .. versionadded:: 3.5.2 - -.. data:: TYPE_CHECKING - - A special constant that is assumed to be ``True`` by 3rd party static - type checkers. It is ``False`` at runtime. Usage:: - - if TYPE_CHECKING: - import expensive_mod - - def fun(arg: 'expensive_mod.SomeType') -> None: - local_var: expensive_mod.AnotherType = other_fun() - - Note that the first type annotation must be enclosed in quotes, making it a - "forward reference", to hide the ``expensive_mod`` reference from the - interpreter runtime. Type annotations for local variables are not - evaluated, so the second annotation does not need to be enclosed in quotes. - - .. versionadded:: 3.5.2 - Functions and decorators -"""""""""""""""""""""""" +------------------------ .. function:: cast(typ, val) @@ -1504,8 +1501,13 @@ Functions and decorators assert isinstance(open('/some/file'), Closable) - **Warning:** this will check only the presence of the required methods, - not their type signatures! + .. note:: + + :func:`runtime_checkable` will check only the presence of the required methods, + not their type signatures! For example, :class:`builtins.complex ` + implements :func:`__float__`, therefore it passes an :func:`issubclass` check + against :class:`SupportsFloat`. However, the ``complex.__float__`` method + exists only to raise a :class:`TypeError` with a more informative message. .. versionadded:: 3.8 @@ -1526,3 +1528,26 @@ Functions and decorators Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public. + + +Constant +-------- + +.. data:: TYPE_CHECKING + + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: + + if TYPE_CHECKING: + import expensive_mod + + def fun(arg: 'expensive_mod.SomeType') -> None: + local_var: expensive_mod.AnotherType = other_fun() + + Note that the first type annotation must be enclosed in quotes, making it a + "forward reference", to hide the ``expensive_mod`` reference from the + interpreter runtime. Type annotations for local variables are not + evaluated, so the second annotation does not need to be enclosed in quotes. + + .. versionadded:: 3.5.2 + From 826340a376f0a25ffb6bfc588da342f11697cfcd Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Sun, 2 Aug 2020 17:09:11 -0300 Subject: [PATCH 17/18] bpo-40979: reordering after second review --- Doc/library/typing.rst | 916 +++++++++++++++++++++-------------------- 1 file changed, 466 insertions(+), 450 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 6bbcb9fd6de384..1d6ee84f9f9303 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -473,89 +473,73 @@ Special forms These can be used as types in annotations using ``[]``, each having a unique syntax. -.. data:: Annotated +.. data:: Tuple - A type, introduced in :pep:`593` (``Flexible function and variable - annotations``), to decorate existing types with context-specific metadata - (possibly multiple pieces of it, as ``Annotated`` is variadic). - Specifically, a type ``T`` can be annotated with metadata ``x`` via the - typehint ``Annotated[T, x]``. This metadata can be used for either static - analysis or at runtime. If a library (or tool) encounters a typehint - ``Annotated[T, x]`` and has no special logic for metadata ``x``, it - should ignore it and simply treat the type as ``T``. Unlike the - ``no_type_check`` functionality that currently exists in the ``typing`` - module which completely disables typechecking annotations on a function - or a class, the ``Annotated`` type allows for both static typechecking - of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) - together with runtime access to ``x`` within a specific application. + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. - Ultimately, the responsibility of how to interpret the annotations (if - at all) is the responsibility of the tool or library encountering the - ``Annotated`` type. A tool or library encountering an ``Annotated`` type - can scan through the annotations to determine if they are of interest - (e.g., using ``isinstance()``). + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. - When a tool or a library does not support annotations or encounters an - unknown annotation it should just ignore it and treat annotated type as - the underlying type. + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - It's up to the tool consuming the annotations to decide whether the - client is allowed to have several annotations on one type and how to - merge those annotations. +.. data:: Union - Since the ``Annotated`` type allows you to put several annotations of - the same (or different) type(s) on any node, the tools or libraries - consuming those annotations are in charge of dealing with potential - duplicates. For example, if you are doing value range analysis you might - allow this:: + Union type; ``Union[X, Y]`` means either X or Y. - T1 = Annotated[int, ValueRange(-10, 5)] - T2 = Annotated[T1, ValueRange(-20, 3)] + To define a union, use e.g. ``Union[int, str]``. Details: - Passing ``include_extras=True`` to :func:`get_type_hints` lets one - access the extra annotations at runtime. + * The arguments must be types and there must be at least one. - The details of the syntax: + * Unions of unions are flattened, e.g.:: - * The first argument to ``Annotated`` must be a valid type + Union[Union[int, str], float] == Union[int, str, float] - * Multiple type annotations are supported (``Annotated`` supports variadic - arguments):: + * Unions of a single argument vanish, e.g.:: - Annotated[int, ValueRange(3, 10), ctype("char")] + Union[int] == int # The constructor actually returns int - * ``Annotated`` must be called with at least two arguments ( - ``Annotated[int]`` is not valid) + * Redundant arguments are skipped, e.g.:: - * The order of the annotations is preserved and matters for equality - checks:: + Union[int, str, int] == Union[int, str] - Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ - int, ctype("char"), ValueRange(3, 10) - ] + * When comparing unions, the argument order is ignored, e.g.:: - * Nested ``Annotated`` types are flattened, with metadata ordered - starting with the innermost annotation:: + Union[int, str] == Union[str, int] - Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ - int, ValueRange(3, 10), ctype("char") - ] + * You cannot subclass or instantiate a union. - * Duplicated annotations are not removed:: + * You cannot write ``Union[X][Y]``. - Annotated[int, ValueRange(3, 10)] != Annotated[ - int, ValueRange(3, 10), ValueRange(3, 10) - ] + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. - * ``Annotated`` can be used with nested and generic aliases:: + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. - T = TypeVar('T') - Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] - V = Vec[int] +.. data:: Optional - V == Annotated[List[Tuple[int, int]], MaxLen(10)] + Optional type. - .. versionadded:: 3.9 + ``Optional[X]`` is equivalent to ``Union[X, None]``. + + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ``Optional`` qualifier on its type + annotation just because it is optional. For example:: + + def foo(arg: int = 0) -> None: + ... + + On the other hand, if an explicit value of ``None`` is allowed, the + use of ``Optional`` is appropriate, whether the argument is optional + or not. For example:: + + def foo(arg: Optional[int] = None) -> None: + ... .. data:: Callable @@ -574,6 +558,70 @@ These can be used as types in annotations using ``[]``, each having a unique syn ``Callable[..., Any]``, and in turn to :class:`collections.abc.Callable`. +.. class:: Type(Generic[CT_co]) + + A variable annotated with ``C`` may accept a value of type ``C``. In + contrast, a variable annotated with ``Type[C]`` may accept values that are + classes themselves -- specifically, it will accept the *class object* of + ``C``. For example:: + + a = 3 # Has type 'int' + b = int # Has type 'Type[int]' + c = type(a) # Also has type 'Type[int]' + + Note that ``Type[C]`` is covariant:: + + class User: ... + class BasicUser(User): ... + class ProUser(User): ... + class TeamUser(User): ... + + # Accepts User, BasicUser, ProUser, TeamUser, ... + def make_new_user(user_class: Type[User]) -> User: + # ... + return user_class() + + The fact that ``Type[C]`` is covariant implies that all subclasses of + ``C`` should implement the same constructor signature and class method + signatures as ``C``. The type checker should flag violations of this, + but should also allow constructor calls in subclasses that match the + constructor calls in the indicated base class. How the type checker is + required to handle this particular case may change in future revisions of + :pep:`484`. + + The only legal parameters for :class:`Type` are classes, :data:`Any`, + :ref:`type variables `, and unions of any of these types. + For example:: + + def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... + + ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent + to ``type``, which is the root of Python's metaclass hierarchy. + + .. versionadded:: 3.5.2 + +.. data:: Literal + + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent to + the provided literal (or one of several literals). For example:: + + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... + + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... + + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker + + ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value + is allowed as type argument to ``Literal[...]``, but type checkers may + impose restrictions. See :pep:`586` for more details about literal types. + + .. versionadded:: 3.8 + .. data:: ClassVar Special type construct to mark class variables. @@ -619,224 +667,311 @@ These can be used as types in annotations using ``[]``, each having a unique syn .. versionadded:: 3.8 -.. data:: Literal +.. data:: Annotated - A type that can be used to indicate to type checkers that the - corresponding variable or function parameter has a value equivalent to - the provided literal (or one of several literals). For example:: + A type, introduced in :pep:`593` (``Flexible function and variable + annotations``), to decorate existing types with context-specific metadata + (possibly multiple pieces of it, as ``Annotated`` is variadic). + Specifically, a type ``T`` can be annotated with metadata ``x`` via the + typehint ``Annotated[T, x]``. This metadata can be used for either static + analysis or at runtime. If a library (or tool) encounters a typehint + ``Annotated[T, x]`` and has no special logic for metadata ``x``, it + should ignore it and simply treat the type as ``T``. Unlike the + ``no_type_check`` functionality that currently exists in the ``typing`` + module which completely disables typechecking annotations on a function + or a class, the ``Annotated`` type allows for both static typechecking + of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) + together with runtime access to ``x`` within a specific application. - def validate_simple(data: Any) -> Literal[True]: # always returns True - ... + Ultimately, the responsibility of how to interpret the annotations (if + at all) is the responsibility of the tool or library encountering the + ``Annotated`` type. A tool or library encountering an ``Annotated`` type + can scan through the annotations to determine if they are of interest + (e.g., using ``isinstance()``). - MODE = Literal['r', 'rb', 'w', 'wb'] - def open_helper(file: str, mode: MODE) -> str: - ... + When a tool or a library does not support annotations or encounters an + unknown annotation it should just ignore it and treat annotated type as + the underlying type. - open_helper('/some/path', 'r') # Passes type check - open_helper('/other/path', 'typo') # Error in type checker + It's up to the tool consuming the annotations to decide whether the + client is allowed to have several annotations on one type and how to + merge those annotations. - ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value - is allowed as type argument to ``Literal[...]``, but type checkers may - impose restrictions. See :pep:`586` for more details about literal types. + Since the ``Annotated`` type allows you to put several annotations of + the same (or different) type(s) on any node, the tools or libraries + consuming those annotations are in charge of dealing with potential + duplicates. For example, if you are doing value range analysis you might + allow this:: - .. versionadded:: 3.8 + T1 = Annotated[int, ValueRange(-10, 5)] + T2 = Annotated[T1, ValueRange(-20, 3)] -.. data:: Optional + Passing ``include_extras=True`` to :func:`get_type_hints` lets one + access the extra annotations at runtime. - Optional type. + The details of the syntax: - ``Optional[X]`` is equivalent to ``Union[X, None]``. + * The first argument to ``Annotated`` must be a valid type - Note that this is not the same concept as an optional argument, - which is one that has a default. An optional argument with a - default does not require the ``Optional`` qualifier on its type - annotation just because it is optional. For example:: + * Multiple type annotations are supported (``Annotated`` supports variadic + arguments):: - def foo(arg: int = 0) -> None: - ... + Annotated[int, ValueRange(3, 10), ctype("char")] - On the other hand, if an explicit value of ``None`` is allowed, the - use of ``Optional`` is appropriate, whether the argument is optional - or not. For example:: + * ``Annotated`` must be called with at least two arguments ( + ``Annotated[int]`` is not valid) - def foo(arg: Optional[int] = None) -> None: - ... + * The order of the annotations is preserved and matters for equality + checks:: -.. data:: Tuple + Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ + int, ctype("char"), ValueRange(3, 10) + ] - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. + * Nested ``Annotated`` types are flattened, with metadata ordered + starting with the innermost annotation:: - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. + Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ + int, ValueRange(3, 10), ctype("char") + ] - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + * Duplicated annotations are not removed:: -.. class:: Type(Generic[CT_co]) + Annotated[int, ValueRange(3, 10)] != Annotated[ + int, ValueRange(3, 10), ValueRange(3, 10) + ] - A variable annotated with ``C`` may accept a value of type ``C``. In - contrast, a variable annotated with ``Type[C]`` may accept values that are - classes themselves -- specifically, it will accept the *class object* of - ``C``. For example:: + * ``Annotated`` can be used with nested and generic aliases:: - a = 3 # Has type 'int' - b = int # Has type 'Type[int]' - c = type(a) # Also has type 'Type[int]' + T = TypeVar('T') + Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] + V = Vec[int] - Note that ``Type[C]`` is covariant:: + V == Annotated[List[Tuple[int, int]], MaxLen(10)] - class User: ... - class BasicUser(User): ... - class ProUser(User): ... - class TeamUser(User): ... + .. versionadded:: 3.9 - # Accepts User, BasicUser, ProUser, TeamUser, ... - def make_new_user(user_class: Type[User]) -> User: - # ... - return user_class() +Building generic types +"""""""""""""""""""""" - The fact that ``Type[C]`` is covariant implies that all subclasses of - ``C`` should implement the same constructor signature and class method - signatures as ``C``. The type checker should flag violations of this, - but should also allow constructor calls in subclasses that match the - constructor calls in the indicated base class. How the type checker is - required to handle this particular case may change in future revisions of - :pep:`484`. +These are not used in annotations. They are building blocks for creating generic types. - The only legal parameters for :class:`Type` are classes, :data:`Any`, - :ref:`type variables `, and unions of any of these types. - For example:: +.. class:: Generic - def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... + Abstract base class for generic types. - ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent - to ``type``, which is the root of Python's metaclass hierarchy. + A generic type is typically declared by inheriting from an + instantiation of this class with one or more type variables. + For example, a generic mapping type might be defined as:: + + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. + + This class can then be used as follows:: + + X = TypeVar('X') + Y = TypeVar('Y') + + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + try: + return mapping[key] + except KeyError: + return default + +.. class:: TypeVar + + Type variable. + + Usage:: + + T = TypeVar('T') # Can be anything + A = TypeVar('A', str, bytes) # Must be str or bytes + + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See class Generic for more + information on generic types. Generic functions work as follows:: + + def repeat(x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n + + def longest(x: A, y: A) -> A: + """Return the longest of two strings.""" + return x if len(x) >= len(y) else y + + The latter example's signature is essentially the overloading + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. + + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issubclass` should not be used with types. + + Type variables may be marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implicitly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. + +.. data:: AnyStr + + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + +.. class:: Protocol(Generic) + + Base class for protocol classes. Protocol classes are defined like this:: - .. versionadded:: 3.5.2 + class Proto(Protocol): + def meth(self) -> int: + ... -.. data:: Union + Such classes are primarily used with static type checkers that recognize + structural subtyping (static duck-typing), for example:: - Union type; ``Union[X, Y]`` means either X or Y. + class C: + def meth(self) -> int: + return 0 - To define a union, use e.g. ``Union[int, str]``. Details: + def func(x: Proto) -> int: + return x.meth() - * The arguments must be types and there must be at least one. + func(C()) # Passes static type check - * Unions of unions are flattened, e.g.:: + See :pep:`544` for details. Protocol classes decorated with + :func:`runtime_checkable` (described later) act as simple-minded runtime + protocols that check only the presence of given attributes, ignoring their + type signatures. - Union[Union[int, str], float] == Union[int, str, float] + Protocol classes can be generic, for example:: - * Unions of a single argument vanish, e.g.:: + class GenProto(Protocol[T]): + def meth(self) -> T: + ... - Union[int] == int # The constructor actually returns int + .. versionadded:: 3.8 - * Redundant arguments are skipped, e.g.:: +.. decorator:: runtime_checkable - Union[int, str, int] == Union[int, str] + Mark a protocol class as a runtime protocol. - * When comparing unions, the argument order is ignored, e.g.:: + Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. + This raises :exc:`TypeError` when applied to a non-protocol class. This + allows a simple-minded structural check, very similar to "one trick ponies" + in :mod:`collections.abc` such as :class:`Iterable`. For example:: - Union[int, str] == Union[str, int] + @runtime_checkable + class Closable(Protocol): + def close(self): ... - * You cannot subclass or instantiate a union. + assert isinstance(open('/some/file'), Closable) - * You cannot write ``Union[X][Y]``. + .. note:: - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + :func:`runtime_checkable` will check only the presence of the required methods, + not their type signatures! For example, :class:`builtins.complex ` + implements :func:`__float__`, therefore it passes an :func:`issubclass` check + against :class:`SupportsFloat`. However, the ``complex.__float__`` method + exists only to raise a :class:`TypeError` with a more informative message. - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. + .. versionadded:: 3.8 Other special directives """""""""""""""""""""""" -These are not used in annotations. They are building blocks used for declaring types. +These are not used in annotations. They are building blocks for declaring types. -.. class:: ForwardRef +.. class:: NamedTuple - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. + Typed version of :func:`collections.namedtuple`. -.. class:: Generic + Usage:: - Abstract base class for generic types. + class Employee(NamedTuple): + name: str + id: int - A generic type is typically declared by inheriting from an - instantiation of this class with one or more type variables. - For example, a generic mapping type might be defined as:: + This is equivalent to:: - class Mapping(Generic[KT, VT]): - def __getitem__(self, key: KT) -> VT: - ... - # Etc. + Employee = collections.namedtuple('Employee', ['name', 'id']) - This class can then be used as follows:: + To give a field a default value, you can assign to it in the class body:: - X = TypeVar('X') - Y = TypeVar('Y') + class Employee(NamedTuple): + name: str + id: int = 3 - def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: - try: - return mapping[key] - except KeyError: - return default + employee = Employee('Guido') + assert employee.id == 3 -.. function:: NewType(name, tp) + Fields with a default value must come after any fields without a default. - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: + The resulting class has an extra attribute ``__annotations__`` giving a + dict that maps the field names to the field types. (The field names are in + the ``_fields`` attribute and the default values are in the + ``_field_defaults`` attribute both of which are part of the namedtuple + API.) - UserId = NewType('UserId', int) - first_user = UserId(1) + ``NamedTuple`` subclasses can also have docstrings and methods:: - .. versionadded:: 3.5.2 + class Employee(NamedTuple): + """Represents an employee.""" + name: str + id: int = 3 -.. class:: Protocol(Generic) + def __repr__(self) -> str: + return f'' - Base class for protocol classes. Protocol classes are defined like this:: + Backward-compatible usage:: - class Proto(Protocol): - def meth(self) -> int: - ... + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) - Such classes are primarily used with static type checkers that recognize - structural subtyping (static duck-typing), for example:: + .. versionchanged:: 3.6 + Added support for :pep:`526` variable annotation syntax. - class C: - def meth(self) -> int: - return 0 + .. versionchanged:: 3.6.1 + Added support for default values, methods, and docstrings. - def func(x: Proto) -> int: - return x.meth() + .. versionchanged:: 3.8 + The ``_field_types`` and ``__annotations__`` attributes are + now regular dictionaries instead of instances of ``OrderedDict``. - func(C()) # Passes static type check + .. versionchanged:: 3.9 + Removed the ``_field_types`` attribute in favor of the more + standard ``__annotations__`` attribute which has the same information. - See :pep:`544` for details. Protocol classes decorated with - :func:`runtime_checkable` (described later) act as simple-minded runtime - protocols that check only the presence of given attributes, ignoring their - type signatures. +.. function:: NewType(name, tp) - Protocol classes can be generic, for example:: + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: - class GenProto(Protocol[T]): - def meth(self) -> T: - ... + UserId = NewType('UserId', int) + first_user = UserId(1) - .. versionadded:: 3.8 + .. versionadded:: 3.5.2 .. class:: TypedDict(dict) - A simple typed namespace. At runtime it is - a plain :class:`dict`. + Special construct to add type hints to a dictionary. + At runtime it is a plain :class:`dict`. - ``TypedDict`` creates a dictionary type that expects all of its + ``TypedDict`` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. @@ -877,62 +1012,6 @@ These are not used in annotations. They are building blocks used for declaring t .. versionadded:: 3.8 -.. class:: TypeVar - - Type variable. - - Usage:: - - T = TypeVar('T') # Can be anything - A = TypeVar('A', str, bytes) # Must be str or bytes - - Type variables exist primarily for the benefit of static type - checkers. They serve as the parameters for generic types as well - as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows:: - - def repeat(x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n - - def longest(x: A, y: A) -> A: - """Return the longest of two strings.""" - return x if len(x) >= len(y) else y - - The latter example's signature is essentially the overloading - of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note - that if the arguments are instances of some subclass of :class:`str`, - the return type is still plain :class:`str`. - - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. - - Type variables may be marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. Alternatively, - a type variable may specify an upper bound using ``bound=``. - This means that an actual type substituted (explicitly or implicitly) - for the type variable must be a subclass of the boundary type, - see :pep:`484`. - -Predefined type variable -"""""""""""""""""""""""" - -.. data:: AnyStr - - ``AnyStr`` is a type variable defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. - - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: - - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b - - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - Generic concrete collections ---------------------------- @@ -977,103 +1056,43 @@ Corresponding to built-in types A generic version of :class:`builtins.frozenset `. -.. note:: :data:`Tuple` is a special form. - -Corresponding to types in :mod:`collections` -"""""""""""""""""""""""""""""""""""""""""""" - -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.defaultdict`. - - .. versionadded:: 3.5.2 - -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - - A generic version of :class:`collections.OrderedDict`. - - .. versionadded:: 3.7.2 - -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) - - A generic version of :class:`collections.ChainMap`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: Counter(collections.Counter, Dict[T, int]) - - A generic version of :class:`collections.Counter`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: Deque(deque, MutableSequence[T]) - - A generic version of :class:`collections.deque`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 - -.. class:: NamedTuple - - Typed version of :func:`collections.namedtuple`. - - Usage:: - - class Employee(NamedTuple): - name: str - id: int - - This is equivalent to:: +.. note:: :data:`Tuple` is a special form. - Employee = collections.namedtuple('Employee', ['name', 'id']) +Corresponding to types in :mod:`collections` +"""""""""""""""""""""""""""""""""""""""""""" - To give a field a default value, you can assign to it in the class body:: +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - class Employee(NamedTuple): - name: str - id: int = 3 + A generic version of :class:`collections.defaultdict`. - employee = Employee('Guido') - assert employee.id == 3 + .. versionadded:: 3.5.2 - Fields with a default value must come after any fields without a default. +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - The resulting class has an extra attribute ``__annotations__`` giving a - dict that maps the field names to the field types. (The field names are in - the ``_fields`` attribute and the default values are in the - ``_field_defaults`` attribute both of which are part of the namedtuple - API.) + A generic version of :class:`collections.OrderedDict`. - ``NamedTuple`` subclasses can also have docstrings and methods:: + .. versionadded:: 3.7.2 - class Employee(NamedTuple): - """Represents an employee.""" - name: str - id: int = 3 +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) - def __repr__(self) -> str: - return f'' + A generic version of :class:`collections.ChainMap`. - Backward-compatible usage:: + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 - Employee = NamedTuple('Employee', [('name', str), ('id', int)]) +.. class:: Counter(collections.Counter, Dict[T, int]) - .. versionchanged:: 3.6 - Added support for :pep:`526` variable annotation syntax. + A generic version of :class:`collections.Counter`. - .. versionchanged:: 3.6.1 - Added support for default values, methods, and docstrings. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 - .. versionchanged:: 3.8 - The ``_field_types`` and ``__annotations__`` attributes are - now regular dictionaries instead of instances of ``OrderedDict``. +.. class:: Deque(deque, MutableSequence[T]) - .. versionchanged:: 3.9 - Removed the ``_field_types`` attribute in favor of the more - standard ``__annotations__`` attribute which has the same information. + A generic version of :class:`collections.deque`. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 Other concrete types """""""""""""""""""" @@ -1085,7 +1104,7 @@ Other concrete types Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` and ``BinaryIO(IO[bytes])`` represent the types of I/O streams such as returned by - :func:`open`. These types are in the ``typing.io`` namespace. + :func:`open`. These types are also in the ``typing.io`` namespace. .. class:: Pattern Match @@ -1095,7 +1114,7 @@ Other concrete types :func:`re.match`. These types (and the corresponding functions) are generic in ``AnyStr`` and can be made specific by writing ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. These types are in the ``typing.re`` namespace. + ``Match[bytes]``. These types are also in the ``typing.re`` namespace. .. class:: Text @@ -1185,13 +1204,6 @@ Corresponding to collections in :mod:`collections.abc` Corresponding to other types in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" -.. class:: ContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 - .. class:: Iterable(Generic[T_co]) A generic version of :class:`collections.abc.Iterable`. @@ -1261,13 +1273,6 @@ Asynchronous programming .. versionadded:: 3.5.3 -.. class:: AsyncContextManager(Generic[T_co]) - - A generic version of :class:`contextlib.AbstractAsyncContextManager`. - - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 - .. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) An async generator can be annotated by the generic type @@ -1319,6 +1324,25 @@ Asynchronous programming .. versionadded:: 3.5.2 + +Context manager types +""""""""""""""""""""" + +.. class:: ContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractContextManager`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 + +.. class:: AsyncContextManager(Generic[T_co]) + + A generic version of :class:`contextlib.AbstractAsyncContextManager`. + + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 + + Protocols --------- @@ -1368,6 +1392,34 @@ Functions and decorators runtime we intentionally don't check anything (we want this to be as fast as possible). +.. decorator:: overload + + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + :exc:`NotImplementedError`. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + + + See :pep:`484` for details and comparison with other typing semantics. + .. decorator:: final A decorator to indicate to type checkers that the decorated method @@ -1393,24 +1445,43 @@ Functions and decorators .. versionadded:: 3.8 -.. function:: get_args(tp) -.. function:: get_origin(tp) +.. decorator:: no_type_check - Provide basic introspection for generic types and special typing forms. + Decorator to indicate that annotations are not type hints. - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) + This mutates the function(s) in place. - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) +.. decorator:: no_type_check_decorator - .. versionadded:: 3.8 + Decorator to give another decorator the :func:`no_type_check` effect. + + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. + +.. decorator:: type_check_only + + Decorator to mark a class or function to be unavailable at runtime. + + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: + + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... + + def fetch_response() -> Response: ... + + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. + +Introspection helpers +--------------------- .. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) @@ -1441,94 +1512,31 @@ Functions and decorators .. versionchanged:: 3.9 Added ``include_extras`` parameter as part of :pep:`593`. -.. decorator:: no_type_check - - Decorator to indicate that annotations are not type hints. - - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). - - This mutates the function(s) in place. - -.. decorator:: no_type_check_decorator - - Decorator to give another decorator the :func:`no_type_check` effect. - - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. - -.. decorator:: overload - - The ``@overload`` decorator allows describing functions and methods - that support multiple different combinations of argument types. A series - of ``@overload``-decorated definitions must be followed by exactly one - non-``@overload``-decorated definition (for the same function/method). - The ``@overload``-decorated definitions are for the benefit of the - type checker only, since they will be overwritten by the - non-``@overload``-decorated definition, while the latter is used at - runtime but should be ignored by a type checker. At runtime, calling - a ``@overload``-decorated function directly will raise - :exc:`NotImplementedError`. An example of overload that gives a more - precise type than can be expressed using a union or a type variable:: - - @overload - def process(response: None) -> None: - ... - @overload - def process(response: int) -> Tuple[int, str]: - ... - @overload - def process(response: bytes) -> str: - ... - def process(response): - - - See :pep:`484` for details and comparison with other typing semantics. - -.. decorator:: runtime_checkable - - Mark a protocol class as a runtime protocol. - - Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. - This raises :exc:`TypeError` when applied to a non-protocol class. This - allows a simple-minded structural check, very similar to "one trick ponies" - in :mod:`collections.abc` such as :class:`Iterable`. For example:: +.. function:: get_args(tp) +.. function:: get_origin(tp) - @runtime_checkable - class Closable(Protocol): - def close(self): ... + Provide basic introspection for generic types and special typing forms. - assert isinstance(open('/some/file'), Closable) + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: - .. note:: + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) - :func:`runtime_checkable` will check only the presence of the required methods, - not their type signatures! For example, :class:`builtins.complex ` - implements :func:`__float__`, therefore it passes an :func:`issubclass` check - against :class:`SupportsFloat`. However, the ``complex.__float__`` method - exists only to raise a :class:`TypeError` with a more informative message. + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) .. versionadded:: 3.8 -.. decorator:: type_check_only - - Decorator to mark a class or function to be unavailable at runtime. - - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: - - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... - - def fetch_response() -> Response: ... - - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. +.. class:: ForwardRef + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. Constant -------- @@ -1544,10 +1552,18 @@ Constant def fun(arg: 'expensive_mod.SomeType') -> None: local_var: expensive_mod.AnotherType = other_fun() - Note that the first type annotation must be enclosed in quotes, making it a + The first type annotation must be enclosed in quotes, making it a "forward reference", to hide the ``expensive_mod`` reference from the interpreter runtime. Type annotations for local variables are not evaluated, so the second annotation does not need to be enclosed in quotes. + .. note:: + + If ``from __future__ import annotations`` is used in Python 3.7 or later, + annotations are not evaluated at function definition time. + Instead, the are stored as strings in ``__annotations__``, + This makes it unnecessary to use quotes around the annotation. + (see :pep:`563`). + .. versionadded:: 3.5.2 From 7273e95a192d4fb31c4dab03934f6243cb4da82f Mon Sep 17 00:00:00 2001 From: Luciano Ramalho Date: Sun, 2 Aug 2020 18:49:49 -0300 Subject: [PATCH 18/18] bpo-40979: added PEP 585 deprecation notes --- Doc/library/typing.rst | 129 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 128 insertions(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 1d6ee84f9f9303..44b537f1669e1a 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -438,7 +438,25 @@ can define new custom protocols to fully enjoy structural subtyping Module contents =============== -The module defines the following classes, functions and decorators: +The module defines the following classes, functions and decorators. + +.. note:: + + This module defines several types that are subclasses of pre-existing + standard library classes which also extend :class:`Generic` + to support type variables inside ``[]``. + These types became redundant in Python 3.9 when the + corresponding pre-existing classes were enhanced to support ``[]``. + + The redundant types are deprecated as of Python 3.9 but no + deprecation warnings will be issued by the interpreter. + It is expected that type checkers will flag the deprecated types + when the checked program targets Python 3.9 or newer. + + The deprecated types will be removed from the :mod:`typing` module + in the first Python version released 5 years after the release of Python 3.9.0. + See details in :pep:`585`—*Type Hinting Generics In Standard Collections*. + Special typing primitives ------------------------- @@ -487,6 +505,9 @@ These can be used as types in annotations using ``[]``, each having a unique syn use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. + .. deprecated:: 3.9 + :class:`builtins.tuple ` now supports ``[]``. See :pep:`585`. + .. data:: Union Union type; ``Union[X, Y]`` means either X or Y. @@ -558,6 +579,9 @@ These can be used as types in annotations using ``[]``, each having a unique syn ``Callable[..., Any]``, and in turn to :class:`collections.abc.Callable`. + .. deprecated:: 3.9 + :class:`collections.abc.Callable` now supports ``[]``. See :pep:`585`. + .. class:: Type(Generic[CT_co]) A variable annotated with ``C`` may accept a value of type ``C``. In @@ -600,6 +624,9 @@ These can be used as types in annotations using ``[]``, each having a unique syn .. versionadded:: 3.5.2 + .. deprecated:: 3.9 + :class:`builtins.type ` now supports ``[]``. See :pep:`585`. + .. data:: Literal A type that can be used to indicate to type checkers that the @@ -1029,6 +1056,9 @@ Corresponding to built-in types def count_words(text: str) -> Dict[str, int]: ... + .. deprecated:: 3.9 + :class:`builtins.dict ` now supports ``[]``. See :pep:`585`. + .. class:: List(list, MutableSequence[T]) Generic version of :class:`list`. @@ -1046,16 +1076,25 @@ Corresponding to built-in types def keep_positives(vector: Sequence[T]) -> List[T]: return [item for item in vector if item > 0] + .. deprecated:: 3.9 + :class:`builtins.list ` now supports ``[]``. See :pep:`585`. + .. class:: Set(set, MutableSet[T]) A generic version of :class:`builtins.set `. Useful for annotating return types. To annotate arguments it is preferred to use an abstract collection type such as :class:`AbstractSet`. + .. deprecated:: 3.9 + :class:`builtins.set ` now supports ``[]``. See :pep:`585`. + .. class:: FrozenSet(frozenset, AbstractSet[T_co]) A generic version of :class:`builtins.frozenset `. + .. deprecated:: 3.9 + :class:`builtins.frozenset ` now supports ``[]``. See :pep:`585`. + .. note:: :data:`Tuple` is a special form. Corresponding to types in :mod:`collections` @@ -1067,12 +1106,18 @@ Corresponding to types in :mod:`collections` .. versionadded:: 3.5.2 + .. deprecated:: 3.9 + :class:`collections.defaultdict` now supports ``[]``. See :pep:`585`. + .. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) A generic version of :class:`collections.OrderedDict`. .. versionadded:: 3.7.2 + .. deprecated:: 3.9 + :class:`collections.OrderedDict` now supports ``[]``. See :pep:`585`. + .. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) A generic version of :class:`collections.ChainMap`. @@ -1080,6 +1125,9 @@ Corresponding to types in :mod:`collections` .. versionadded:: 3.5.4 .. versionadded:: 3.6.1 + .. deprecated:: 3.9 + :class:`collections.ChainMap` now supports ``[]``. See :pep:`585`. + .. class:: Counter(collections.Counter, Dict[T, int]) A generic version of :class:`collections.Counter`. @@ -1087,6 +1135,9 @@ Corresponding to types in :mod:`collections` .. versionadded:: 3.5.4 .. versionadded:: 3.6.1 + .. deprecated:: 3.9 + :class:`collections.Counter` now supports ``[]``. See :pep:`585`. + .. class:: Deque(deque, MutableSequence[T]) A generic version of :class:`collections.deque`. @@ -1094,6 +1145,9 @@ Corresponding to types in :mod:`collections` .. versionadded:: 3.5.4 .. versionadded:: 3.6.1 + .. deprecated:: 3.9 + :class:`collections.deque` now supports ``[]``. See :pep:`585`. + Other concrete types """""""""""""""""""" @@ -1116,6 +1170,9 @@ Other concrete types ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or ``Match[bytes]``. These types are also in the ``typing.re`` namespace. + .. deprecated:: 3.9 + Classes ``Pattern`` and ``Match`` from :mod:`re` now support ``[]``. See :pep:`585`. + .. class:: Text ``Text`` is an alias for ``str``. It is provided to supply a forward @@ -1140,6 +1197,9 @@ Corresponding to collections in :mod:`collections.abc` A generic version of :class:`collections.abc.Set`. + .. deprecated:: 3.9 + :class:`collections.abc.Set` now supports ``[]``. See :pep:`585`. + .. class:: ByteString(Sequence[int]) A generic version of :class:`collections.abc.ByteString`. @@ -1150,24 +1210,39 @@ Corresponding to collections in :mod:`collections.abc` As a shorthand for this type, :class:`bytes` can be used to annotate arguments of any of the types mentioned above. + .. deprecated:: 3.9 + :class:`collections.abc.ByteString` now supports ``[]``. See :pep:`585`. + .. class:: Collection(Sized, Iterable[T_co], Container[T_co]) A generic version of :class:`collections.abc.Collection` .. versionadded:: 3.6.0 + .. deprecated:: 3.9 + :class:`collections.abc.Collection` now supports ``[]``. See :pep:`585`. + .. class:: Container(Generic[T_co]) A generic version of :class:`collections.abc.Container`. + .. deprecated:: 3.9 + :class:`collections.abc.Container` now supports ``[]``. See :pep:`585`. + .. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) A generic version of :class:`collections.abc.ItemsView`. + .. deprecated:: 3.9 + :class:`collections.abc.ItemsView` now supports ``[]``. See :pep:`585`. + .. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) A generic version of :class:`collections.abc.KeysView`. + .. deprecated:: 3.9 + :class:`collections.abc.KeysView` now supports ``[]``. See :pep:`585`. + .. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) A generic version of :class:`collections.abc.Mapping`. @@ -1176,30 +1251,50 @@ Corresponding to collections in :mod:`collections.abc` def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: return word_list[word] + .. deprecated:: 3.9 + :class:`collections.abc.Mapping` now supports ``[]``. See :pep:`585`. + .. class:: MappingView(Sized, Iterable[T_co]) A generic version of :class:`collections.abc.MappingView`. + .. deprecated:: 3.9 + :class:`collections.abc.MappingView` now supports ``[]``. See :pep:`585`. + .. class:: MutableMapping(Mapping[KT, VT]) A generic version of :class:`collections.abc.MutableMapping`. + .. deprecated:: 3.9 + :class:`collections.abc.MutableMapping` now supports ``[]``. See :pep:`585`. + .. class:: MutableSequence(Sequence[T]) A generic version of :class:`collections.abc.MutableSequence`. + .. deprecated:: 3.9 + :class:`collections.abc.MutableSequence` now supports ``[]``. See :pep:`585`. + .. class:: MutableSet(AbstractSet[T]) A generic version of :class:`collections.abc.MutableSet`. + .. deprecated:: 3.9 + :class:`collections.abc.MutableSet` now supports ``[]``. See :pep:`585`. + .. class:: Sequence(Reversible[T_co], Collection[T_co]) A generic version of :class:`collections.abc.Sequence`. + .. deprecated:: 3.9 + :class:`collections.abc.Sequence` now supports ``[]``. See :pep:`585`. + .. class:: ValuesView(MappingView[VT_co]) A generic version of :class:`collections.abc.ValuesView`. + .. deprecated:: 3.9 + :class:`collections.abc.ValuesView` now supports ``[]``. See :pep:`585`. Corresponding to other types in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" @@ -1208,10 +1303,16 @@ Corresponding to other types in :mod:`collections.abc` A generic version of :class:`collections.abc.Iterable`. + .. deprecated:: 3.9 + :class:`collections.abc.Iterable` now supports ``[]``. See :pep:`585`. + .. class:: Iterator(Iterable[T_co]) A generic version of :class:`collections.abc.Iterator`. + .. deprecated:: 3.9 + :class:`collections.abc.Iterator` now supports ``[]``. See :pep:`585`. + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -1243,6 +1344,9 @@ Corresponding to other types in :mod:`collections.abc` yield start start += 1 + .. deprecated:: 3.9 + :class:`collections.abc.Generator` now supports ``[]``. See :pep:`585`. + .. class:: Hashable An alias to :class:`collections.abc.Hashable` @@ -1251,6 +1355,9 @@ Corresponding to other types in :mod:`collections.abc` A generic version of :class:`collections.abc.Reversible`. + .. deprecated:: 3.9 + :class:`collections.abc.Reversible` now supports ``[]``. See :pep:`585`. + .. class:: Sized An alias to :class:`collections.abc.Sized` @@ -1273,6 +1380,9 @@ Asynchronous programming .. versionadded:: 3.5.3 + .. deprecated:: 3.9 + :class:`collections.abc.Coroutine` now supports ``[]``. See :pep:`585`. + .. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) An async generator can be annotated by the generic type @@ -1306,24 +1416,36 @@ Asynchronous programming .. versionadded:: 3.6.1 + .. deprecated:: 3.9 + :class:`collections.abc.AsyncGenerator` now supports ``[]``. See :pep:`585`. + .. class:: AsyncIterable(Generic[T_co]) A generic version of :class:`collections.abc.AsyncIterable`. .. versionadded:: 3.5.2 + .. deprecated:: 3.9 + :class:`collections.abc.AsyncIterable` now supports ``[]``. See :pep:`585`. + .. class:: AsyncIterator(AsyncIterable[T_co]) A generic version of :class:`collections.abc.AsyncIterator`. .. versionadded:: 3.5.2 + .. deprecated:: 3.9 + :class:`collections.abc.AsyncIterator` now supports ``[]``. See :pep:`585`. + .. class:: Awaitable(Generic[T_co]) A generic version of :class:`collections.abc.Awaitable`. .. versionadded:: 3.5.2 + .. deprecated:: 3.9 + :class:`collections.abc.Awaitable` now supports ``[]``. See :pep:`585`. + Context manager types """"""""""""""""""""" @@ -1335,6 +1457,9 @@ Context manager types .. versionadded:: 3.5.4 .. versionadded:: 3.6.0 + .. deprecated:: 3.9 + :class:`collections.contextlib.AbstractContextManager` now supports ``[]``. See :pep:`585`. + .. class:: AsyncContextManager(Generic[T_co]) A generic version of :class:`contextlib.AbstractAsyncContextManager`. @@ -1342,6 +1467,8 @@ Context manager types .. versionadded:: 3.5.4 .. versionadded:: 3.6.2 + .. deprecated:: 3.9 + :class:`collections.contextlib.AbstractAsyncContextManager` now supports ``[]``. See :pep:`585`. Protocols ---------