AI Engineering
/
Production & Reliability
/

Implement a bounded retry policy

Production & Reliability · 2 to 3 hours

Implement a bounded retry policy

Implement a pure, deterministic retry policy with attempt limits, capped jittered backoff, retry-after, and a deadline check, then fit it into an executor that handles cancellation, partial streams, and uncertain writes.

Medium
Hands-on

3 h

Used in interview rounds:

Live Coding (Hard)

Step 1: The question

Implement the retry policy and reproduce the fixture decisions. Explain how it fits into a deadline-aware executor, how SDK retries are accounted for, and what changes after a partial streamed response or an uncertain write.

Define the retry contract

The teaching policy permits at most three total attempts. It retries only explicitly classified transient failures when replay is safe. It uses capped exponential backoff with full jitter and respects a normalized retry-after delay as a minimum wait.

Before scheduling a retry, it requires enough remaining time for the delay plus a configured minimum useful attempt window. This check does not guarantee that the next request will finish. The runtime must still enforce the actual deadline.

The provider's raw Retry-After value may be a duration or date depending on the protocol. Parsing and validation happen before this pure policy function receives a nonnegative delay in milliseconds.

A pure retry policy

The policy is a pure function. The caller passes the failure classification, the number of attempts already made (including the one that just failed), the time left before the task deadline, and a random value for jitter. Nothing inside it sleeps, reads a clock, or generates randomness, so every decision is reproducible.

type Failure = {
  transient: boolean;
  safeToRetry: boolean;
  retryAfterMs?: number;
};
type Policy = {
  maxAttempts: number;
  baseDelayMs: number;
  maxDelayMs: number;
  minimumAttemptMs: number;
};
type Decision =
  | { retry: true; delayMs: number }
  | { retry: false; reason: "unsafe" | "permanent" | "attempts" | "deadline" };

// Throws on a negative or non-finite duration, an attempt count that is not a
// positive integer, a random value outside [0, 1), or a zero minimumAttemptMs.
// Then stops, in order, for unsafe replay, permanent failure, and used-up attempts.
// Otherwise:
//   ceiling = min(maxDelayMs, baseDelayMs * 2 ** (completedAttempts - 1))
//   delayMs = max(floor(random * ceiling), retryAfterMs ?? 0)
// and stops for "deadline" when delayMs + minimumAttemptMs > remainingMs.
declare function decideRetry(
  failure: Failure,
  completedAttempts: number,
  remainingMs: number,
  random: number,
  policy: Policy
): Decision;

const policy: Policy = {
  maxAttempts: 3, baseDelayMs: 200, maxDelayMs: 1000, minimumAttemptMs: 300
};

Constraints

  • The policy is pure and synchronous: the caller passes the remaining time and a random value in [0, 1)
  • At most three total attempts per logical task, counting SDK and proxy attempts
  • Retry only failures that are both transient and safe to replay
  • Never wait or retry past the task deadline

Deliverables

  • A retry policy that passes the fixture decisions in the linked practice problem
  • The executor loop: deadline, timeouts, waiting, and cancellation
  • How SDK and proxy retries fit into the same budget
  • What happens after a partial streamed response and after an uncertain write

Step 2: Hints

Optional. Each hint gives a little more away.

Step 3: Work through it

Code exercise

Bounded retry policy

Implement decideRetry so each fixture returns its expected decision. Validate the inputs, stop for unsafe replay, permanent failure, and used-up attempts in that order, then compute the capped, jittered delay with retry-after as a minimum, and stop when the delay plus the minimum attempt window does not fit in the remaining time.

Expected

Every fixture in the next exercise returns its expected decision. The reference programs log the decisions after one, two, and three failed attempts with random 0.5, which match the first three rows.

Open coding workspace

Limitations

The function accepts normalized, trusted policy values and an already-classified failure, so validate raw provider responses and external configuration before calling it. It does not sleep, dispatch requests, cancel operations, or reconcile writes. Passing its deadline check does not guarantee that the next attempt finishes: the runtime must still enforce the actual deadline.

Predict the outcome

Predict the fixture decisions

Every case uses the policy above (maxAttempts 3, baseDelayMs 200, maxDelayMs 1000, minimumAttemptMs 300) and random value 0.5. Predict each decision before you reveal it.

Failure and stateRemaining timeExpected decisionWhy
Transient, safe, one completed attempt2,000 ms??
Transient, safe, two completed attempts2,000 ms??
Transient, safe, three completed attempts2,000 ms??
Permanent failure, safe replay2,000 ms??
Transient but unsafe replay2,000 ms??
Transient, safe, retry-after 700 ms, one completed attempt2,000 ms??
Same retry-after case900 ms??
Transient, safe, one completed attempt (the first retry needs 100 + 300 ms)399 ms??

Kept in this browser. Saving your score below keeps a copy with it.

0 / 5000

Step 4: Reference answer

Step 5: Score yourself

Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.

Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.

Failure classification and replay safety
Delay calculations
Attempt and deadline enforcement
Cancellation and stream handling
Fallback integration

0 of 10 · Scored 0 of 5 dimensions

Ready at 8 of 10, with no 0 in Failure classification and replay safety

Kept in this browser until you sign up, then it moves to your account.

Complete this level when the fixtures pass and you can explain exactly when a new request is dispatched, denied, cancelled, or routed to a fallback.

Your saved scores

Practice saves and mock interview answers on this question, newest first.

No saved scores yet. Score every dimension, then save.

Step 6: Follow-up questions

An interviewer may ask these next.

Why use a pure retry policy?

Does backoff guarantee recovery?

Why honor retry-after?

What if a timeout leaves the request running?

Can fallback reset the deadline?

Step 7: Mark complete