Tags

Testing

Testing

0 lessons
1 problem
6 AI Engineering questions
6 community items

testing

Practice Problems

1 problem

Evaluation Scoring Harness

Free
Not Started
Medium

Score an AI evaluation set: task success, answer coverage, and answered accuracy with their own denominators, per-slice results, and missing outputs counted instead of hidden.

AI Engineering
LLM Evaluation
Testing
Hash Map / Dictionary
Intermediate

11

AI Engineering

6 questions
AI Engineering

Validate a model's JSON review summary

Write a pure function that accepts a model's JSON review summary or returns every contract violation as a stable error code, predict its result for eight replies, and say what a valid summary still cannot prove.

ai-engineering
structured-output
error-handling
testing

6

Easy
AI Engineering

Extend a tool adapter with a search tool

Extend the validated ticket adapter with a paginated search_tickets tool behind a tool registry under a 3-hour cap, hand in tests and a README, then review a sample submission as the interviewer would.

ai-engineering
tool-calling
pagination
multi-tenant
testing

5

Medium
AI Engineering

Build and score a small evaluation set

Score a six-case evaluation set with a deterministic harness, report success, coverage, and answered accuracy with named denominators, and keep missing outputs visible in the totals.

ai-engineering
llm-evaluation
testing
observability
debugging

7

Medium
AI Engineering

Prove a context selector cannot leak data

Implement a context selector that enforces tenant, group, and deletion rules before ranking, then design tests proving that forbidden records and internal notes never reach the model-bound payload.

ai-engineering
data-privacy
authorization
multi-tenant
security
testing

6

Medium
AI Engineering

Explain train, validation, and test splits

Explain what training, validation, and test splits are each for, then repair a fictional marketplace's evaluation plan by choosing a split that matches how its model will be used.

ai-engineering
machine-learning
data-leakage
testing

6

Easy
AI Engineering

Diagnose leakage and misleading metrics

Implement an audit that flags leaky features, preprocessing, time splits, selection, and mismatched reports, predict its findings on five fixture experiments, and diagnose why a churn model that looked excellent offline failed after launch.

ai-engineering
machine-learning
data-leakage
debugging
testing

8

Medium

Community

6 items
Code Snippet

Why I Stopped Mocking `fetch` and Reached for MSW

The two-handler MSW setup I drop into every Vitest project. Pattern-matched URL routing, typed JSON responses, and per-test overrides without re-importing the server.

TypeScript
testing
http
code-template
utility

1k

26

4.4 (9)

May 8, 2026

by @ezb1981

Code Snippet

Freeze Time and Fast-Forward in Jest

The clock-control playbook I keep on a sticky note: freeze `Date.now`, advance pending timers without sleeping, and write tests for debounce/throttle helpers that actually finish in milliseconds.

JavaScript
testing
unit-testing
code-template

203

2

4.6 (8)

May 2, 2026

by @rohanbakr

Code Snippet

The pytest Fixture Builder I Cannot Live Without

A four-stage tour of the fixture-builder pattern: a callable factory that mints test models with sensible defaults, layered overrides, deterministic IDs, and per-test isolation. The shape I paste into every conftest.py.

Python
testing
unit-testing
code-template
utility

700

2

4.3 (14)

Apr 30, 2026

by @yunatorres

Article

Testing Pyramid vs Trophy: Pick the Right Shape

Most teams ship the testing pyramid by accident. The trophy is what actually matches modern frontend work. Here is how to choose.

testing
unit-testing
craftsmanship
reliability
code-organization

654

9

4.2 (14)

Mar 4, 2026

by @aishasantos

Code Snippet

How I Stopped My Jest Snapshots From Churning

A serializer that strips dates, ids, and request hashes out of snapshot JSON so a fresh git clone does not nuke the snapshot diff. Drop into `snapshotSerializers` and forget.

JavaScript
testing
unit-testing
utility
code-template

379

12

4.2 (11)

Dec 14, 2025

by @marcusreddy

Article

Dependency Injection Without a Framework

DI is a pattern, not a framework. The plain-language version (pass things in instead of newing them inside) covers ninety percent of cases. The trade-off versus DI containers, and where each one breaks down.

design-patterns
patterns
oop
abstraction
testing

499

5

4.3 (14)

Dec 4, 2025

by @ramijohansson