Steven's Knowledge

Data Model & Metaprogramming

The object model that powers Python -- dunder methods, descriptors, __init_subclass__, metaclasses, and the ABC/Protocol hierarchy

Data Model & Metaprogramming

Everything in Python is an object, and the data model -- the dunder methods the interpreter calls on your behalf -- is the protocol you implement to extend the language. This page goes from operator overloading down to descriptors, metaclasses, and abstract base classes: the machinery behind ORMs, frameworks, and the standard library itself.

The Data Model and Dunder Methods

Everything in Python is an object, and the "dunder" (double-underscore) methods are the protocol the interpreter calls on your behalf. Mastering them is what separates someone who uses Python from someone who can extend it.

class Money:
    __slots__ = ("amount", "currency")  # no __dict__: less memory, no typos

    def __init__(self, amount: int, currency: str):
        self.amount = amount      # store cents as int -- never float for money
        self.currency = currency

    def __eq__(self, other: object) -> bool:
        if not isinstance(other, Money):
            return NotImplemented   # let Python try the reflected op
        return (self.amount, self.currency) == (other.amount, other.currency)

    def __hash__(self) -> int:
        return hash((self.amount, self.currency))  # define with __eq__ or lose hashability

    def __add__(self, other: "Money") -> "Money":
        if self.currency != other.currency:
            raise ValueError("currency mismatch")
        return Money(self.amount + other.amount, self.currency)

    def __repr__(self) -> str:
        return f"Money({self.amount}, {self.currency!r})"  # !r for debuggability

Two rules experts internalise: defining __eq__ without __hash__ makes instances unhashable (you can no longer put them in a set or dict key); and returning NotImplemented (the singleton, not raising NotImplementedError) lets Python fall back to the other operand's reflected method.

Descriptors

Descriptors are the machinery behind @property, methods, classmethod, and ORM fields. A descriptor is any object defining __get__/__set__. __set_name__ (3.6+) gives it the attribute name for free:

class Validated:
    def __set_name__(self, owner, name: str):
        self.private = f"_{name}"

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return getattr(obj, self.private)

    def __set__(self, obj, value):
        if value < 0:
            raise ValueError("must be non-negative")
        setattr(obj, self.private, value)

class Account:
    balance = Validated()  # reusable, declarative validation across many fields

Hooking Subclass Creation

__init_subclass__ runs whenever a class is subclassed -- a lightweight, readable alternative to a metaclass for registries and plugin systems:

class Plugin:
    registry: dict[str, type] = {}

    def __init_subclass__(cls, *, key: str, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.registry[key] = cls

class CsvExporter(Plugin, key="csv"): ...
class JsonExporter(Plugin, key="json"): ...
# Plugin.registry == {"csv": CsvExporter, "json": JsonExporter}

Metaclasses

A metaclass is the class of a class. Reach for one only when __init_subclass__ and class decorators are not enough -- they are powerful but make code harder to follow. The canonical legitimate uses are frameworks: Django's Model, SQLAlchemy's declarative base, and ABCs all use metaclasses.

class Singleton(type):
    _instances: dict = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Database(metaclass=Singleton):
    def __init__(self):
        self.pool = create_pool()

assert Database() is Database()  # same instance every time

Rule of thumb: if you are asking whether you need a metaclass, you almost certainly do not. Prefer class decorators or __init_subclass__.

Abstract Base Classes and the Protocol Hierarchy

Python has two ways to express "this type supports an interface". ABCs (abc.ABC) are nominal -- you explicitly inherit and the class cannot be instantiated until every @abstractmethod is implemented. This is the right tool for a base class your own code controls:

from abc import ABC, abstractmethod

class PaymentGateway(ABC):
    @abstractmethod
    def charge(self, cents: int) -> str: ...

    def refund(self, charge_id: str) -> None:    # concrete methods are allowed
        raise NotImplementedError

class StripeGateway(PaymentGateway):
    def charge(self, cents: int) -> str:
        return "ch_123"
# Forgetting to implement charge() makes StripeGateway() raise at construction.

collections.abc defines the protocols the language itself uses -- Iterable, Iterator, Sequence, Mapping, Hashable, and so on. Inheriting from them gives you mixin methods for free: implement __getitem__ and __len__ on a Sequence and you get __contains__, __iter__, __reversed__, index, and count automatically.

from collections.abc import Sequence

class Page(Sequence):
    def __init__(self, items): self._items = items
    def __getitem__(self, i): return self._items[i]
    def __len__(self): return len(self._items)
    # in, iter(), reversed(), .index(), .count() all now work -- no extra code.

The distinction that matters: use typing.Protocol (structural, from the typing section) when you want duck typing checked statically with no inheritance; use ABCs (nominal) when you want a runtime-enforced contract and shared implementation. Protocols describe what callers need; ABCs anchor a hierarchy you own.

On this page