`Self` type isn't considered to be the same as the explicit class type

[emilyyyylime]

Bug Report
In methods where the Self type (from typing) is used, the Self type from is considered as distinct to the actual self type (say, Foo). This is the cause of a multitude of issues.
To exemplify these issues, I will use enum.Enums. as they have a clear reason to need access to the runtime self type.
In a method returning Self, one may not return Foo.Variant, receiving an error along the lines of:

file.py:#: error: Incompatible return value type (got "Foo", expected "Self")  [return-value]

To work around this, one must use typing.cast or # type: ignore comments.

Furthermore, within those same methods returning Self, the self parameter auto-magically assumes the type of Self, to accommodate the fact that Self is seen by mypy as a TypeVar and "A function returning TypeVar should receive at least one argument containing the same TypeVar [type-var]".
This brings on the unfortunate consequence that one cannot match the self value against its own enum variants exhaustively, mypy simply refuses to acknowledge that self may take on values of Foo.Variant. Trying to access the fields through the self parameter as self.Variant reveals that it indeed has the type auto (or whichever value was assigned to it), rather than the literal or the class.

To Reproduce

https://mypy-play.net/?mypy=latest&python=3.11&flags=strict&gist=808ead69249e64d6929a55149b0a3f1c

Actual Behavior

For the above example, the errors include:

main.py:35: error: Missing return statement  [return]
main.py:37: error: Incompatible return value type (got "Dir", expected "Self")  [return-value]
main.py:38: error: Incompatible return value type (got "Dir", expected "Self")  [return-value]
main.py:39: error: Incompatible return value type (got "Dir", expected "Self")  [return-value]
main.py:40: error: Incompatible return value type (got "Dir", expected "Self")  [return-value]

Environment

• Mypy version used: master (1.8.0+dev.5b1a231425ac807b7118aac6a68b633949412a36)
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files): strict = True
• Python version used: 3.11.5

[AlexWaygood]

I believe this is a duplicate of #14075

[erictraut]

I don't think this is the same as #14075. I also don't think this is a bug. Mypy is doing the right thing here.

Self is specifically not the same as the explicit class type. If your intent is to specify the explicit class type, you should use that type. Self is effectively a type variable that has an upper bound of the current class.

Consider the following code:

class Parent:
    def method1(self) -> Self:
        return Parent() # This is a type violation

class Child(Parent):
    pass

Child().method1() # This should return an instance of Child

For details, refer to PEP 673.

Here's the corrected code sample that type checks without error:

from enum import Enum, auto

class Dir(Enum):
    LEFT = auto()
    UP = auto()
    RIGHT = auto()
    DOWN = auto()

    def opposed(self) -> "Dir":
        match self:
            case Dir.LEFT:
                return Dir.RIGHT
            case Dir.UP:
                return Dir.DOWN
            case Dir.RIGHT:
                return Dir.LEFT
            case Dir.DOWN:
                return Dir.UP

[AlexWaygood]

Right, it's not the same thing as #14075, and I see the point you're making @erictraut. I think it would be reasonable for type checkers to exclude final classes (including enums with members) from this kind of check, however. No subclass of Dir can exist, so there's no practical difference between Self and the explicit class type here

[emilyyyylime]

I will say, I forgot about putting the type in a string 😅

[erictraut]

I think it would be reasonable for type checkers to exclude final classes from this kind of check

IMO that would require a change to the typing spec, and I'm not convinced it would be a good change. I think it would be better (both more correct from a typing standpoint and clearer in the code) to use the class itself in the annotation rather than Self in cases like this.

@AlexWaygood, if you feel strongly about this, please raise it in the typing forums to see if others think that exemptions should be added to Self type compatibility rules for classes that are either explicitly or implicitly final. If you don't feel strongly about it, then it might be best to simply close this issue.

[jace]

I've found inconsistencies in the way Self is handled between Pyright and Mypy (reported in #16554). PEP 673 doesn't cover both the scenarios I've found (decorator and unbound method). Mypy gets it wrong in the first, Pyright in the second.

[hauntsaninja]

I generally agree with Eric here.

Although I do find the following related example interesting. mypy will complain about the second one being able to return None, but not the first. This is something that mypy should fix, and will be important if we make the change in #14075. (pyright complains about neither, presumably because it's being cleverer about enum finality)

from __future__ import annotations
from enum import Enum, auto
from typing import Self

class Dir(Enum):
    LEFT  = auto()
    UP    = auto()
    RIGHT = auto()
    DOWN  = auto()

    def mypy_is_fine_with_this(self) -> Dir:
        match self:
            case Dir.LEFT:  return Dir.RIGHT
            case Dir.UP:    return Dir.DOWN
            case Dir.RIGHT: return Dir.LEFT
            case Dir.DOWN:  return Dir.UP
    
    def mypy_is_worried_this_match_might_not_be_exhaustive(self: Self) -> Dir:
        match self:
            case Dir.LEFT:  return Dir.RIGHT
            case Dir.UP:    return Dir.DOWN
            case Dir.RIGHT: return Dir.LEFT
            case Dir.DOWN:  return Dir.UP

[jpgoldberg]

I am not sure if this is the appropriate topic-self-types Types for self issue add this comment to, but I haven't seen my particular and very simple encounter with the problem mentioned elsewhere.

from typing import Self
class Point:
    def __init__(self, x: float, y: float) -> None:
        self._x = x
        self._y = y

    def copy(self: Self) -> Self:
        p = Point(self._x, self._y)
        return p  # This fails

mypy (1.11.2) tells me

self-type.py:9: error: Incompatible return value type (got "Point", expected "Self")  [return-value]

I will leave it to others who have a far better understanding than I do of what Self is supposed to mean and how it is implemented to chat about what is going on. I just saw that this was an on-going discussion, and wanted to add a distilled minimal example of where I am seeing what looks like a problem to me.

[erictraut]

@jpgoldberg, the code in your code sample is a type violation, and mypy is correct to report this as an error. The type of p is Point in your code sample, and this type is not assignable to Self. You can think of Self as a type variable scoped to the class in which it appears. Your code is roughly equivalent to this:

class Point:
    ...
    def copy[S: Point](self: S) -> S:
        p = Point(self._x, self._y)
        return p  # Type violation

[brianschubert]

@jpgoldberg Mypy is correct to reject that code, see #16558 (comment) above. The issue is that Point.copy always returns a Point, even in subclasses where Self will be a different type. To make that type-check, you’d need to use something like:

class Point:
    # ...
    def copy(self: Self) -> Self:
        p = type(self)(self._x, self._y)
        return p

[jpgoldberg]

Thank you @brianschubert!

[...] To make that type-check, you’d need to use something like:

...
    def copy(self: Self) -> Self:
        p = type(self)(self._x, self._y)
        return p

I had at one point attempted p = Self(self._x, self._y), knowing full well that that it wouldn't work, as Self is not a class; but my attempt reflects what I wanted. Your suggestion is very much in the spirit of what I was seeking.