INTERNAL ERROR when using mypy with pre-commit (all versions)

[TravisHardman]

Crash Report

Mypy was working fine on my django project until about a week ago, when I started getting this error. I'm not sure what caused it. My python code used to be in the root directory of my project and now I moved it to the backend/ directory, and the problem started around then, but I'm not sure whether that is related.

I am using pre-commit and have mypy set up in my .pre-commit-config.yaml. The error actually only happens when I run mypy as part of pre-commit. If I just run mypy . from the command line, it works fine.

Presently, whenever I run pre-commit, I get an INTERNAL ERROR (details below).

Traceback

/home/travis/Repos/bethpage/backend/bethpage/models.py:23: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.16.0+dev.23e2d0f8cbaa2f4874d8f6a6bee3756922c78407
Traceback (most recent call last):
  File "/home/travis/Repos/bethpage/.tox/py312/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/main.py", line 119, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/main.py", line 203, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 191, in build
    result = _build(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 267, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 2937, in dispatch
    process_graph(graph, manager)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 3335, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 3430, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 94, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 252, in process_functions
    process_top_level_function(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 291, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 351, in semantic_analyze_target
    analyzer.refresh_partial(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 655, in refresh_partial
    self.accept(node)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 7254, in accept
    node.accept(self)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/nodes.py", line 811, in accept
    return visitor.visit_func_def(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 913, in visit_func_def
    self.analyze_func_def(defn)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 958, in analyze_func_def
    self.defer(defn)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 6900, in defer
    assert not self.final_iteration, "Must not defer during final iteration"
           ^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Must not defer during final iteration
/home/travis/Repos/bethpage/backend/bethpage/models.py:23: : note: use --pdb to drop into pdb

To Reproduce

This crash happens any time I change any file in my backend/ directory and then run git add -A; pre-commit.

It does not happen if I just run cd backend; mypy .

If I use the --no-incremental flag or delete the cache directory, the problem still persists.

If I open up backend/bethpage/models.py and add a line that just says "pass" somewhere above line 23 (the line cited in the error), then run git add -A; pre-commit, it actually works fine and completes without errors. The next time I need to commit something, I'll get the error again, but I can just remove that "pass" line and once again, the error goes away.

Here is the beginning of that /backend/bethpage/models.py file:

import uuid
from typing import (
    Any,
    ClassVar,
    Dict,
    Generic,
    Iterable,
    List,
    Mapping,
    Optional,
    Self,
    Tuple,
    TypeVar,
)

from django.db import models, transaction
from django.db.models.fields.reverse_related import ForeignObjectRel
from django.utils import timezone

T = TypeVar("T", bound="BaseModel")


class BaseQuerySet(models.QuerySet[T], Generic[T]):

    def bulk_create(self, objs: Iterable[T], *args: Any, **kwargs: Any) -> List[T]:
        for obj in objs:
            obj.full_clean()
        return super().bulk_create(objs, *args, **kwargs)
...etc...

Your Environment

• Mypy version used: I have tried versions of mypy from 1.0.0 to 1.14.1 and have also tried using the master branch on github, always with the same error.
• Python version used: 3.12
• Operating system: Arch Linux

.pre-commit-config.yaml

repos:
  - repo: local
    hooks:
      - id: mypy
        name: mypy
        entry: mypy
        language: system
        types: [python]
        args:
          - --config-file=backend/mypy.ini
          - --show-traceback
...etc...

mypy.ini

[mypy]
python_version = 3.12
mypy_path = backend
plugins =
  mypy_django_plugin.main,
  mypy_drf_plugin.main
strict = True
show_error_codes = True
disallow_untyped_calls = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
disallow_any_unimported = True
check_untyped_defs = True
no_implicit_optional = True
warn_unused_ignores = True
warn_return_any = True
warn_unreachable = True

[mypy.plugins.django-stubs]
django_settings_module = bethpage.settings

[A5rocks]

What happens if you disable the plugins?

And does pre-commit run -a fail?

[TravisHardman]

pre-commit run -a does not work. I get the same error I have in the description above.

Disabling the plugins does work, and mypy no longer returns an error. Specifically, disabling the mypy_django_plugin.main causes it to work. I can keep the mypy_drf_plugin.main plugin and there are no issues.

I updated django-stubs and django-stubs-ext to the latest version but the problem persists. Of course, I do kind of need the django-stubs plugin for my project to work with mypy.

[A5rocks]

If you run mypy <list of files> --config-file=backend/mypy.ini --show-traceback that should also crash, I think. Could you check?

cc @sobolevn

[TravisHardman]

Actually no. It still works when running mypy directly.

(py312) [travis@bananas bethpage]$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   backend/bethpage/settings.py

(py312) [travis@bananas bethpage]$ git add -A
(py312) [travis@bananas bethpage]$ pre-commit
mypy.....................................................................Failed
- hook id: mypy
- exit code: 2

/home/travis/Repos/bethpage/backend/bethpage/models.py:24: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at https://github.com/python/mypy/issues
version: 1.14.1
/home/travis/Repos/bethpage/backend/bethpage/models.py:24: : note: please use --show-traceback to print a traceback when reporting a bug

black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
codespell................................................................Passed
ESLint...............................................(no files to check)Skipped
Prettier.............................................(no files to check)Skipped
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed

At this point I made another change in backend/bethpage/settings.py to make sure to bypass any cache created from the prior step, then continued:

(py312) [travis@bananas bethpage]$ git add -A
(py312) [travis@bananas bethpage]$ mypy backend/ --config-file=backend/mypy.ini --show-traceback
Success: no issues found in 80 source files

[A5rocks]

Sorry just to be explicit, I mean passing mypy a list of .py files. IIRC the way pre-commit works is it does that, so I assume that's the cause of the crash. Such as, find -name "*.py" | xargs mypy --config-file=backend/mypy.ini --show-traceback. (maybe I'm wrong about how pre-commit works)

[sobolevn]

Does it happen without django-stubs enabled?

[TravisHardman]

@A5rocks The issue still does not happen when passing specific files to mypy. The problem does happen with pre-commit, but then I run directly with the list of files and get:

(py312) [travis@bananas bethpage]$ find backend/ -name "*.py" | xargs mypy --config-file=backend/mypy.ini --show-traceback
Success: no issues found in 80 source files

So it is still limited only to pre-commit as far as I can tell.

@sobolevn When I remove django-stubs, the error does not happen, even with pre-commit. It does seem to be tied specifically to pre-commit and specifically to django-stubs. For clarity, earlier post where I checked that: #18579 (comment)

[sterliakov]

This is indeed likely specific to django-stubs, but has nothing to do with pre-commit.

Full script:

cd "$(mktemp -d)"
uv venv --python 3.12 && . ./.venv/bin/activate
uv pip install mypy django django-stubs

mkdir -p backend/bethpage
touch backend/bethpage/__init__.py

cat <<EOF > backend/bethpage/models.py
from typing import Generic, TypeVar

from django.db import models

T = TypeVar("T", bound="BaseModel")

class BaseQuerySet(models.QuerySet[T], Generic[T]): pass
EOF

cat <<EOF > backend/bethpage/settings.py
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
INSTALLED_APPS = []
EOF

cat <<EOF > backend/mypy.ini
[mypy]
python_version = 3.12
mypy_path = backend
plugins =
  mypy_django_plugin.main
strict = True

[mypy.plugins.django-stubs]
django_settings_module = bethpage.settings
EOF

mypy backend --config-file backend/mypy.ini --show-traceback

Out:

backend/bethpage/settings.py:4: error: Need type annotation for "INSTALLED_APPS" (hint: "INSTALLED_APPS: list[<type>] = ...")  [var-annotated]
backend/bethpage/models.py:5: error: Name "BaseModel" is not defined  [name-defined]
backend/bethpage/models.py:7: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.14.1
Traceback (most recent call last):
  File "mypy/semanal.py", line 7286, in accept
  File "mypy/nodes.py", line 827, in accept
  File "mypy/semanal.py", line 907, in visit_func_def
  File "mypy/semanal.py", line 952, in analyze_func_def
  File "mypy/semanal.py", line 6940, in defer
AssertionError: Must not defer during final iteration
backend/bethpage/models.py:7: : note: use --pdb to drop into pdb

[sterliakov]

The core of the problem is in unresolved quoted name in typevar bound: replacing BaseModel with something defined or removing the surrounding quotes prevents the crash.

[sobolevn]

PRs to django-stubs are welcome :)