The idioms that define Pythonic code -- decorators, context managers, generators, dataclasses, pattern matching, and the standard-library power tools

Language Features

These are the everyday idioms that separate code that merely runs from code that reads as Pythonic. Decorators and context managers wrap behaviour cleanly, generators process data lazily, dataclasses model structure, pattern matching destructures by shape, and functools/itertools turn common patterns into one-liners.

Decorators Deep Dive

Decorators are functions that wrap other functions. Understanding them unlocks metaprogramming in Python.

Basic Decorator Pattern

import functools
import time
from typing import Callable, TypeVar, ParamSpec

P = ParamSpec("P")
R = TypeVar("R")

def timer(func: Callable[P, R]) -> Callable[P, R]:
    @functools.wraps(func)
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__} took {elapsed:.4f}s")
        return result
    return wrapper

@timer
def fetch_users(limit: int) -> list[dict]:
    # ... database query
    pass

Decorator with Parameters

def retry(max_attempts: int = 3, delay: float = 1.0):
    def decorator(func: Callable[P, R]) -> Callable[P, R]:
        @functools.wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            last_exception = None
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    last_exception = e
                    if attempt < max_attempts - 1:
                        time.sleep(delay * (2 ** attempt))
            raise last_exception
        return wrapper
    return decorator

@retry(max_attempts=5, delay=0.5)
def call_external_api(url: str) -> dict:
    # ... HTTP request that might fail
    pass

Context Managers

Context managers ensure resources are properly acquired and released. The with statement is their interface.

from contextlib import contextmanager, asynccontextmanager
from typing import AsyncGenerator

@contextmanager
def database_transaction(connection):
    tx = connection.begin()
    try:
        yield tx
        tx.commit()
    except Exception:
        tx.rollback()
        raise

# Usage
with database_transaction(conn) as tx:
    tx.execute("INSERT INTO users ...")
    tx.execute("INSERT INTO audit_log ...")
    # Commits on success, rolls back on exception

# Async version
@asynccontextmanager
async def managed_connection(pool) -> AsyncGenerator:
    conn = await pool.acquire()
    try:
        yield conn
    finally:
        await pool.release(conn)

Generators and Iterators

Generators produce values lazily, one at a time. They are essential for processing large datasets without loading everything into memory:

def read_large_file(path: str, chunk_size: int = 8192):
    """Read a file in chunks without loading it all into memory."""
    with open(path, "rb") as f:
        while chunk := f.read(chunk_size):
            yield chunk

def batch(iterable, size: int):
    """Group items into fixed-size batches."""
    batch = []
    for item in iterable:
        batch.append(item)
        if len(batch) == size:
            yield batch
            batch = []
    if batch:
        yield batch

# Process millions of records in batches of 1000
for record_batch in batch(read_csv_rows("huge.csv"), 1000):
    bulk_insert(record_batch)

Dataclasses and Pydantic v2

Dataclasses for Internal Data

from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class OrderItem:
    product_id: str
    quantity: int
    unit_price: float

    @property
    def total(self) -> float:
        return self.quantity * self.unit_price

@dataclass
class Order:
    id: str
    customer_id: str
    items: list[OrderItem] = field(default_factory=list)
    created_at: datetime = field(default_factory=datetime.utcnow)

    @property
    def total(self) -> float:
        return sum(item.total for item in self.items)

Pydantic v2 for External Data

Pydantic validates data at the boundary -- API inputs, config files, external service responses. V2 is a ground-up rewrite in Rust, significantly faster:

from pydantic import BaseModel, field_validator, model_validator

class Config(BaseModel):
    database_url: str
    redis_url: str
    max_connections: int = 10
    debug: bool = False

    @field_validator("max_connections")
    @classmethod
    def validate_max_connections(cls, v: int) -> int:
        if v < 1 or v > 100:
            raise ValueError("max_connections must be between 1 and 100")
        return v

    @model_validator(mode="after")
    def validate_debug_settings(self):
        if self.debug and self.max_connections > 5:
            raise ValueError("In debug mode, max_connections should be <= 5")
        return self

Standard Library Power Tools

The expert's productivity comes from knowing functools, itertools, and collections cold.

from functools import lru_cache, cached_property, singledispatch, partial, reduce

# Memoise pure functions -- caches by argument identity
@lru_cache(maxsize=1024)
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)

# Compute once per instance, then cache on the instance __dict__
class Dataset:
    @cached_property
    def stats(self) -> dict:
        return expensive_scan(self.path)  # runs once, even across many reads

# Type-based dispatch -- the Pythonic alternative to isinstance chains
@singledispatch
def serialize(value) -> str:
    raise TypeError(f"unsupported: {type(value)}")

@serialize.register
def _(value: datetime) -> str:
    return value.isoformat()

@serialize.register
def _(value: list) -> str:
    return "[" + ",".join(serialize(v) for v in value) + "]"

itertools is the right tool when you want laziness and constant memory:

import itertools as it

# Lazily flatten, window, and group without building intermediate lists
pairs = it.pairwise([1, 2, 3, 4])          # (1,2), (2,3), (3,4)  -- 3.10+
chunks = it.batched(range(10), 3)          # (0,1,2), (3,4,5), ... -- 3.12+
flat = it.chain.from_iterable(list_of_lists)

# groupby requires pre-sorted input -- the classic gotcha
from operator import itemgetter
rows.sort(key=itemgetter("dept"))
for dept, group in it.groupby(rows, key=itemgetter("dept")):
    process(dept, list(group))   # 'group' is a one-shot iterator

Structural Pattern Matching

match/case (3.10+) is far more than a switch statement -- it destructures by shape, which is ideal for parsing nested data, ASTs, and event payloads:

def handle(event: dict):
    match event:
        case {"type": "click", "target": {"id": id}}:
            return f"clicked {id}"
        case {"type": "key", "key": ("Enter" | "Return")}:
            return "submit"
        case {"type": "resize", "w": int(w), "h": int(h)} if w > 0 and h > 0:
            return f"{w}x{h}"   # capture + type check + guard
        case {"type": t}:
            return f"unknown: {t}"
        case _:
            raise ValueError("malformed event")

The trap: a bare name like case Foo: is a capture pattern that always matches and binds, not an equality test. To match against a constant, use a dotted name (case Color.RED:) or a guard.

Language Features

On this page