Mar 04, 2026
Week 9: How LLMs Work, and Why That Changes How We Test Them
Tokens, temperature, training vs inference. The conceptual foundations that change how you write test assertions for AI systems.
Blog
52 Weeks of GenAI Testing
Recent writing and implementation notes.
9 posts
Feb 25, 2026
Week 8: Fixtures, Parametrization, and Testing at Scale
pytest advanced: fixtures, scopes, parametrization, conftest.py. The tools that make AI evaluation test suites actually work.
Feb 18, 2026
Week 7: pytest Basics and Why Test Discovery Matters
Learning pytest from scratch and applying it to six weeks of accumulated code. Good test naming is documentation.
Feb 11, 2026
Week 6: Error Handling, Logging, and the Unreliable API Problem
Building a robust API caller with retry logic and exponential backoff. LLM APIs fail more than you expect.
Feb 04, 2026
Week 5: File Handling, JSON, and Why JSONL Is the Format for AI Testing
Reading and writing files, JSON serialization, and the JSONL format that underpins most AI evaluation datasets.
Jan 28, 2026
Week 4: Modules, Packages, and Writing Code That Lasts
The week Python scripting became Python engineering. Building a reusable test utilities package from scratch.
Jan 21, 2026
Week 3: Control Flow, Functions, and Validating LLM Responses
Building a validation library for LLM outputs. Deterministic checks for non-deterministic systems.
Jan 14, 2026
Week 2: Data Structures and the List-of-Dicts Pattern
Learning Python data structures by building a test case manager. The list-of-dicts pattern shows up everywhere in AI testing.
Jan 07, 2026
Week 1: Python Basics, Prompt Templates, and a Debugging Rabbit Hole
Starting from zero Python to building a reusable prompt template engine in one week. Including one hour lost to PYTHONPATH.
Week 9: How LLMs Work, and Why That Changes How We Test Them
Tokens, temperature, training vs inference. The conceptual foundations that change how you write test assertions for AI systems.
Mar 04, 2026
Week 8: Fixtures, Parametrization, and Testing at Scale
pytest advanced: fixtures, scopes, parametrization, conftest.py. The tools that make AI evaluation test suites actually work.
Feb 25, 2026
Week 7: pytest Basics and Why Test Discovery Matters
Learning pytest from scratch and applying it to six weeks of accumulated code. Good test naming is documentation.
Feb 18, 2026
Week 6: Error Handling, Logging, and the Unreliable API Problem
Building a robust API caller with retry logic and exponential backoff. LLM APIs fail more than you expect.
Feb 11, 2026
Week 5: File Handling, JSON, and Why JSONL Is the Format for AI Testing
Reading and writing files, JSON serialization, and the JSONL format that underpins most AI evaluation datasets.
Feb 04, 2026
Week 4: Modules, Packages, and Writing Code That Lasts
The week Python scripting became Python engineering. Building a reusable test utilities package from scratch.
Jan 28, 2026
Week 3: Control Flow, Functions, and Validating LLM Responses
Building a validation library for LLM outputs. Deterministic checks for non-deterministic systems.
Jan 21, 2026
Week 2: Data Structures and the List-of-Dicts Pattern
Learning Python data structures by building a test case manager. The list-of-dicts pattern shows up everywhere in AI testing.
Jan 14, 2026
Week 1: Python Basics, Prompt Templates, and a Debugging Rabbit Hole
Starting from zero Python to building a reusable prompt template engine in one week. Including one hour lost to PYTHONPATH.
Jan 07, 2026