Comparing Python Type Checkers: Typing Spec Conformance
pyrefly.org
March 16, 2026
6 min read
🔥🔥🔥🔥🔥
52/100
Summary
Python's type system began with PEP 484, establishing foundational rules for type checking. A comparison of various type checkers reveals differing levels of conformance to the Python typing specification.
Key Takeaways
- The Python typing specification was consolidated to address inconsistencies among various type checkers, including mypy, Pyright, Pytype, and Pyre.
- The typing conformance test suite includes approximately one hundred test files that evaluate type checkers based on expected error reporting.
- As of early March 2026, Pyright has the highest pass rate at 97.8%, while mypy has a pass rate of only 58.3%.
- Pyrefly currently supports all major type system features and aims to improve its conformance rate in the coming months.
Community Sentiment
Positives
- Using the 'ty' type checker has proven to be fast and user-friendly, effectively catching many issues in previously untyped codebases, which enhances code reliability.
- The experimentation with strict lint/type checking settings in Python codebases is a promising approach for improving AI-related projects, indicating a growing focus on code quality in ML applications.
Concerns
- Python's type hints are criticized for being decorative, leading to confusion and potential errors, which undermines the reliability of type checking in AI development.
- Significant gaps in the 'ty' type checker have been identified, with alternatives like 'pyright' catching more issues, raising concerns about the effectiveness of current tools for AI-related coding.