Production & Reliability · 3 to 4 hours
Design SLOs, rollout, and cost controls
Design reliability controls for a multi-tenant AI assistant: SLOs and error budgets, quota-aware admission, fallback and cost reservations, staged rollout with a canary decision, rollback, and recovery.
4 h
Used in interview rounds:
Debugging & Evaluation (Hard)AI System Design (Hard)Take-home & Review (Hard)Overview
Design operational objectives, capacity controls, release procedures, and recovery for a production AI service. A strong answer ties user outcomes to measurable SLOs, bounded resource use, evaluated fallback behavior, and a rollback plan that preserves compatible data and in-flight work.
Study time: 3 to 4 hours
Interview format: System design with calculations
By the end you can
- Define SLIs and SLOs with explicit eligible events, good events, and exclusions, kept separate from semantic quality
- Compute error budgets and burn rates, and design alerts that separate a brief burst from sustained burn
- Estimate quota-bound capacity and enforce admission, isolation, and one retry budget per logical task
- Reserve cost safely across replicas and route only to fallbacks that meet data, schema, quality, and capacity requirements
- Defend a canary decision from cohort-level metrics rather than the aggregate
- Plan rollback, incident response, and disaster recovery that preserve in-flight work and side-effect records
Prerequisites
Helpful background: Comfort with rates, percentages, and basic queueing helps.
This topic runs the application. Evaluation & Observability measures its quality, and Model Serving & Inference covers the runtime underneath.
In this guide
- Define service objectives precisely
- Error budgets and burn rates
- Capacity and quota modeling
- Admission control and isolation
- Fallback architecture and correlated failure
- Cost controls as concurrency-safe state
- Version the complete serving configuration
- Staged rollout and canary evaluation
- Canary data for a release decision
- Work the canary decision
- Rollback and in-flight work
- Incident response and recovery
- Disaster recovery and data integrity
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.
Related on CodeSnatch
Lessons, problems, and articles that build the skills this question uses.
System Design
Shows fast and slow burn-rate alerts and an error budget policy that freezes risky launches when the budget is spent.
System Design
Bulkheads, circuit breakers, graceful degradation, and RTO and RPO for the recovery plan.
System Design
Explains why per-replica counters multiply a limit and how an atomic shared store holds one limit across replicas.
AI Engineering
Sets objectives per workload class and designs admission and overload behavior that protects interactive traffic.
Community article
The expand and contract pattern that keeps schema changes compatible in both directions, so a rollback still works.
Practice the interview question
Answer it in your own words before you look at the reference answer.