Testing
testing
Practice Problems
Evaluation Scoring Harness
Score an AI evaluation set: task success, answer coverage, and answered accuracy with their own denominators, per-slice results, and missing outputs counted instead of hidden.
AI Engineering
Validate a model's JSON review summary
Write a pure function that accepts a model's JSON review summary or returns every contract violation as a stable error code, predict its result for eight replies, and say what a valid summary still cannot prove.
Extend a tool adapter with a search tool
Extend the validated ticket adapter with a paginated search_tickets tool behind a tool registry under a 3-hour cap, hand in tests and a README, then review a sample submission as the interviewer would.
Build and score a small evaluation set
Score a six-case evaluation set with a deterministic harness, report success, coverage, and answered accuracy with named denominators, and keep missing outputs visible in the totals.
Prove a context selector cannot leak data
Implement a context selector that enforces tenant, group, and deletion rules before ranking, then design tests proving that forbidden records and internal notes never reach the model-bound payload.
Explain train, validation, and test splits
Explain what training, validation, and test splits are each for, then repair a fictional marketplace's evaluation plan by choosing a split that matches how its model will be used.
Diagnose leakage and misleading metrics
Implement an audit that flags leaky features, preprocessing, time splits, selection, and mismatched reports, predict its findings on five fixture experiments, and diagnose why a churn model that looked excellent offline failed after launch.
Community
Why I Stopped Mocking `fetch` and Reached for MSW
The two-handler MSW setup I drop into every Vitest project. Pattern-matched URL routing, typed JSON responses, and per-test overrides without re-importing the server.
Freeze Time and Fast-Forward in Jest
The clock-control playbook I keep on a sticky note: freeze `Date.now`, advance pending timers without sleeping, and write tests for debounce/throttle helpers that actually finish in milliseconds.
The pytest Fixture Builder I Cannot Live Without
A four-stage tour of the fixture-builder pattern: a callable factory that mints test models with sensible defaults, layered overrides, deterministic IDs, and per-test isolation. The shape I paste into every conftest.py.
Testing Pyramid vs Trophy: Pick the Right Shape
Most teams ship the testing pyramid by accident. The trophy is what actually matches modern frontend work. Here is how to choose.
How I Stopped My Jest Snapshots From Churning
A serializer that strips dates, ids, and request hashes out of snapshot JSON so a fresh git clone does not nuke the snapshot diff. Drop into `snapshotSerializers` and forget.
Dependency Injection Without a Framework
DI is a pattern, not a framework. The plain-language version (pass things in instead of newing them inside) covers ninety percent of cases. The trade-off versus DI containers, and where each one breaks down.
