Data Model & Metaprogramming
The object model that powers Python -- dunder methods, descriptors, __init_subclass__, metaclasses, and the ABC/Protocol hierarchy
Data Model & Metaprogramming
Everything in Python is an object, and the data model -- the dunder methods the interpreter calls on your behalf -- is the protocol you implement to extend the language. This page goes from operator overloading down to descriptors, metaclasses, and abstract base classes: the machinery behind ORMs, frameworks, and the standard library itself.
The Data Model and Dunder Methods
Everything in Python is an object, and the "dunder" (double-underscore) methods are the protocol the interpreter calls on your behalf. Mastering them is what separates someone who uses Python from someone who can extend it.
class Money:
__slots__ = ("amount", "currency") # no __dict__: less memory, no typos
def __init__(self, amount: int, currency: str):
self.amount = amount # store cents as int -- never float for money
self.currency = currency
def __eq__(self, other: object) -> bool:
if not isinstance(other, Money):
return NotImplemented # let Python try the reflected op
return (self.amount, self.currency) == (other.amount, other.currency)
def __hash__(self) -> int:
return hash((self.amount, self.currency)) # define with __eq__ or lose hashability
def __add__(self, other: "Money") -> "Money":
if self.currency != other.currency:
raise ValueError("currency mismatch")
return Money(self.amount + other.amount, self.currency)
def __repr__(self) -> str:
return f"Money({self.amount}, {self.currency!r})" # !r for debuggabilityTwo rules experts internalise: defining __eq__ without __hash__ makes instances unhashable (you can no longer put them in a set or dict key); and returning NotImplemented (the singleton, not raising NotImplementedError) lets Python fall back to the other operand's reflected method.
Descriptors
Descriptors are the machinery behind @property, methods, classmethod, and ORM fields. A descriptor is any object defining __get__/__set__. __set_name__ (3.6+) gives it the attribute name for free:
class Validated:
def __set_name__(self, owner, name: str):
self.private = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.private)
def __set__(self, obj, value):
if value < 0:
raise ValueError("must be non-negative")
setattr(obj, self.private, value)
class Account:
balance = Validated() # reusable, declarative validation across many fieldsHooking Subclass Creation
__init_subclass__ runs whenever a class is subclassed -- a lightweight, readable alternative to a metaclass for registries and plugin systems:
class Plugin:
registry: dict[str, type] = {}
def __init_subclass__(cls, *, key: str, **kwargs):
super().__init_subclass__(**kwargs)
cls.registry[key] = cls
class CsvExporter(Plugin, key="csv"): ...
class JsonExporter(Plugin, key="json"): ...
# Plugin.registry == {"csv": CsvExporter, "json": JsonExporter}Metaclasses
A metaclass is the class of a class. Reach for one only when __init_subclass__ and class decorators are not enough -- they are powerful but make code harder to follow. The canonical legitimate uses are frameworks: Django's Model, SQLAlchemy's declarative base, and ABCs all use metaclasses.
class Singleton(type):
_instances: dict = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]
class Database(metaclass=Singleton):
def __init__(self):
self.pool = create_pool()
assert Database() is Database() # same instance every timeRule of thumb: if you are asking whether you need a metaclass, you almost certainly do not. Prefer class decorators or __init_subclass__.
Abstract Base Classes and the Protocol Hierarchy
Python has two ways to express "this type supports an interface". ABCs (abc.ABC) are nominal -- you explicitly inherit and the class cannot be instantiated until every @abstractmethod is implemented. This is the right tool for a base class your own code controls:
from abc import ABC, abstractmethod
class PaymentGateway(ABC):
@abstractmethod
def charge(self, cents: int) -> str: ...
def refund(self, charge_id: str) -> None: # concrete methods are allowed
raise NotImplementedError
class StripeGateway(PaymentGateway):
def charge(self, cents: int) -> str:
return "ch_123"
# Forgetting to implement charge() makes StripeGateway() raise at construction.collections.abc defines the protocols the language itself uses -- Iterable, Iterator, Sequence, Mapping, Hashable, and so on. Inheriting from them gives you mixin methods for free: implement __getitem__ and __len__ on a Sequence and you get __contains__, __iter__, __reversed__, index, and count automatically.
from collections.abc import Sequence
class Page(Sequence):
def __init__(self, items): self._items = items
def __getitem__(self, i): return self._items[i]
def __len__(self): return len(self._items)
# in, iter(), reversed(), .index(), .count() all now work -- no extra code.The distinction that matters: use typing.Protocol (structural, from the typing section) when you want duck typing checked statically with no inheritance; use ABCs (nominal) when you want a runtime-enforced contract and shared implementation. Protocols describe what callers need; ABCs anchor a hierarchy you own.