Production & Reliability · 3 to 4 hours
Design SLOs, rollout, and cost controls
Design reliability controls for a multi-tenant AI assistant: SLOs and error budgets, quota-aware admission, fallback and cost reservations, staged rollout with a canary decision, rollback, and recovery.
Hard
System Design
4 h
Premium
Used in interview rounds:
Debugging & Evaluation (Hard)AI System Design (Hard)Take-home & Review (Hard)1
Step 1: The question
Design reliability controls for a multi-tenant AI assistant with streaming, retrieval, and side-effecting tools. Define SLIs and SLOs, error budgets, quota-aware admission, fallback, cost reservations, staged rollout, rollback, and recovery. Decide what to do with the supplied canary.
Constraints
- Several tenants share one assistant that streams answers, retrieves private documents, and calls tools with side effects
- Provider quotas limit tokens and requests per minute, and spend has per-tenant and service-wide caps
- The service runs on several replicas, so limits and budgets must hold across all of them
- Canary data: 40 operational failures in 2,000 canary requests and 90 in 18,000 concurrent control requests; the release policy pauses when the canary's bad-event rate exceeds 1% after at least 1,000 requests
Deliverables
- SLI and SLO definitions: eligible events, good events, exclusions, and a separate quality indicator
- Error-budget and burn-rate alerting approach
- Admission, retry, fallback, and cost-reservation controls, with where each limit is enforced
- Rollout, rollback, and recovery plan covering in-flight work and side effects
- A decision on the supplied canary, with the calculation
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.