Python

Python occupies a unique position: it is both a serious backend language (FastAPI, Django) and the dominant language for data engineering, machine learning, and scripting. A senior engineer who knows Python well can move fluidly between building APIs, writing data pipelines, and automating infrastructure. That versatility is Python's superpower.

The trade-off is performance. Python is slow compared to Go, Java, or Rust. The GIL limits true parallelism. But for the vast majority of backend services, the bottleneck is I/O (database, network), not CPU -- and Python handles I/O-bound work well, especially with async.

Philosophy: Readable, Gradual, Batteries-Included

Python rarely forces a paradigm on you. It is multi-paradigm -- procedural, object-oriented, and functional code all coexist -- and the guiding value is readability: code is read far more often than it is written. Three threads run through the language:

• Gradual everything. Types are optional and added incrementally. Async is opt-in. You can write a five-line script or a strictly-typed, fully-async service in the same language, and grow one into the other.
• The data model is the language. Almost every behaviour -- iteration, context management, operators, attribute access -- is a dunder method you can implement. Mastering it is what separates using Python from extending it.
• Batteries included, ecosystem vast. The standard library is deep, and PyPI covers the rest. The cost is packaging complexity, which modern tools (uv, ruff) have finally tamed.

How This Section Is Organized

These articles go past "you can write it" toward "you understand it" -- the idioms, the runtime behaviour, and the sharp edges that separate senior engineers from mid-level ones:

• Backend Frameworks -- FastAPI for type-driven APIs, Django for batteries-included applications, dependency injection, and middleware
• Language Features -- decorators, context managers, generators, dataclasses and Pydantic, structural pattern matching, and the functools/itertools power tools
• Type System -- gradual typing, structural Protocols, generics (PEP 695), Self, TypedDict, and exhaustiveness checking
• Data Model & Metaprogramming -- dunder methods, descriptors, __init_subclass__, metaclasses, and the ABC/Protocol hierarchy
• Async & Concurrency -- async/await, the asyncio event loop up close, cancellation, TaskGroup, and exception groups
• Runtime & Internals -- CPython bytecode, the GIL and multiprocessing, reference counting, the cyclic GC, GC tuning, and weak references
• Pitfalls & Correctness -- mutable defaults, late-binding closures, is vs ==, and the datetime/Decimal/copy traps that corrupt data silently
• Performance & Native Acceleration -- profiling with cProfile/py-spy, vectorisation, numba/Cython, Rust/PyO3, and zero-copy C interop
• Testing -- pytest fixtures, parametrization, async tests, and mocking
• Tooling & Packaging -- uv, ruff, virtual environments, pyproject.toml distribution, and the import system
• Security -- the pickle/yaml/eval/shell=True remote-code-execution traps and their safe alternatives
• Logging & Observability -- the logging hierarchy, structured JSON logs, contextvars, and OpenTelemetry
• Data Engineering & Databases -- NumPy/Pandas views and copies, SQLAlchemy 2.0 and the N+1 trap, Polars, and ETL

Ecosystem Essentials

Python in New Zealand

Python is one of the most in-demand languages in the NZ market, strong across both backend engineering and data. Nearly every data team -- in fintech, agritech, government analytics, and research -- runs on Python, and the backend engineer who can also write data pipelines is highly valued. The versatility that lets one person own the API, the ETL, and the model is exactly what smaller NZ teams need, which is why "knows Python well" opens doors across the whole stack rather than a single niche.