This PEP proposes adding an @override decorator to the Python type system. This will allow type checkers to prevent a class of bugs that occur when a base class changes methods that are inherited by derived classes.
A primary purpose of type checkers is to flag when refactors or changes break pre-existing semantic structures in the code, so users can identify and make fixes across their project without doing a manual audit of their code.
Python’s type system does not provide a way to identify call sites that need to be changed to stay consistent when an overridden function API changes. This makes refactoring and transforming code more dangerous.
Consider this simple inheritance structure:
If the overridden method on the superclass is renamed or deleted, type checkers will only alert us to update call sites that deal with the base type directly. But the type checker can only see the new code, not the change we made, so it has no way of knowing that we probably also needed to rename the same method on child classes.
A type checker will happily accept this code, even though we are likely introducing bugs:
This code will type check, but there are two potential sources of bugs:
The incorrectly-refactored code is type-safe, but is probably not what we intended and could cause our system to behave incorrectly. The bug can be difficult to track down because our new code likely does execute without throwing exceptions. Tests are less likely to catch the problem, and silent errors can take longer to track down in production.
We are aware of several production outages in multiple typed codebases caused by such incorrect refactors. This is our primary motivation for adding an @override decorator to the type system, which lets developers express the relationship between Parent.foo and Child.foo so that type checkers can detect the problem.
We believe that explicit overrides will make unfamiliar code easier to read than implicit overrides. A developer reading the implementation of a subclass that uses @override can immediately see which methods are overriding functionality in some base class; without this decorator, the only way to quickly find out is using a static analysis tool.
Many popular programming languages support override checks. For example:
Today, there is an Overrides library that provides decorators @overrides [sic] and @final and will enforce them at runtime.
PEP 591 added a @final decorator with the same semantics as those in the Overrides library. But the override component of the runtime library is not supported statically at all, which has added some confusion around the mix/matched support.
Providing support for @override in static checks would add value because:
Using @override will make code more verbose.
When type checkers encounter a method decorated with @typing.override they should treat it as a type error unless that method is overriding a compatible method or attribute in some ancestor class.
The @override decorator should be permitted anywhere a type checker considers a method to be a valid override, which typically includes not only normal methods but also @property, @staticmethod, and @classmethod.
This PEP is exclusively concerned with the handling of the new @override decorator, which specifies that the decorated method must override some attribute in an ancestor class. This PEP does not propose any new rules regarding the type signatures of such methods.
We believe that @override is most useful if checkers also allow developers to opt into a strict mode where methods that override a parent class are required to use the decorator. Strict enforcement should be opt-in for backward compatibility.
The primary reason for a strict mode that requires @override is that developers can only trust that refactors are override-safe if they know that the @override decorator is used throughout the project.
There is another class of bug related to overrides that we can only catch using a strict mode.
Consider the following code:
Imagine we refactor it as follows:
The semantics of our code changed here, which could cause two problems:
At first glance this kind of change may seem unlikely, but it can actually happen often if one or more subclasses have functionality that developers later realize belongs in the base class.
With a strict mode, we will always alert developers when this occurs.
Most of the typed, object-oriented programming languages we looked at have an easy way to require explicit overrides throughout a project:
By default, the @override decorator will be opt-in. Codebases that do not use it will type-check as before, without the additional type safety.
At runtime, @typing.override will make a best-effort attempt to add an attribute __override__ with value True to its argument. By “best-effort” we mean that we will try adding the attribute, but if that fails (for example because the input is a descriptor type with fixed slots) we will silently return the argument as-is.
This is exactly what the @typing.final decorator does, and the motivation is similar: it gives runtime libraries the ability to use @override. As a concrete example, a runtime library could check __override__ in order to automatically populate the __doc__ attribute of child class methods using the parent method docstring.
As described above, adding __override__ may fail at runtime, in which case we will simply return the argument as-is.
In addition, even in cases where it does work, it may be difficult for users to correctly work with multiple decorators, because successfully ensuring the __override__ attribute is set on the final output requires understanding the implementation of each decorator:
As a result, runtime support for setting __override__ is best effort only, and we do not expect type checkers to validate the ordering of decorators.
Modern Integrated Development Environments (IDEs) often provide the ability to automatically update subclasses when renaming a method. But we view this as insufficient for several reasons:
We considered having @typing.override enforce override safety at runtime, similarly to how @overrides.overrides does today.
We rejected this for four reasons:
We considered including a class decorator @require_explicit_overrides, which would have provided a way for base classes to declare that all subclasses must use the @override decorator on method overrides. The Overrides library has a mixin class, EnforceExplicitOverrides, which provides similar behavior in runtime checks.
We decided against this because we expect owners of large codebases will benefit most from @override, and for these use cases having a strict mode where explicit @override is required (see the Backward Compatibility section) provides more benefits than a way to mark base classes.
Moreover we believe that authors of projects who do not consider the extra type safety to be worth the additional boilerplate of using @override should not be forced to do so. Having an optional strict mode puts the decision in the hands of project owners, whereas the use of @require_explicit_overrides in libraries would force project owners to use @override even if they prefer not to.
We considered allowing the caller of @override to specify a specific ancestor class where the overridden method should be defined:
This could be useful for code readability because it makes the override structure more explicit for deep inheritance trees. It also might catch bugs by prompting developers to check that the implementation of an override still makes sense whenever a method being overridden moves from one base class to another.
We decided against it because:
Pyre: A proof of concept is implemented in Pyre:
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0698.rst
Last modified: 2024-06-11 22:12:09 UTC