Requirements

Functional Requirements

1. Browse and search: customers browse a catalog of 100M SKUs with category filters, faceted search, and free-text search.
2. Product detail: photos, variants (size, color), price, availability, reviews, recommendations.
3. Cart: add, update quantity, remove items; cart persists across sessions and devices.
4. Checkout: select shipping address, payment method, delivery option; place the order.
5. Order tracking: customers see status (placed, paid, shipped, delivered) and tracking number.
6. Inventory accuracy: never sell a unit that does not exist; if the last one is taken, the next customer sees out of stock within seconds.
7. Returns and refunds: customers can request a return; once received, payment is refunded.

Out of Scope (state explicitly)

• Seller onboarding and seller-side dashboards.
• Detailed recommendation algorithms (we mention the surface, not the model).
• Warehouse-level robotics and physical fulfillment.
• Tax calculation engine (treated as an external service call).

Non-Functional Requirements

1. Browse latency: p99 < 300 ms for product detail and search.
2. Checkout latency: p99 < 2 s end to end (network + reservation + payment).
3. Availability: 99.99% for browse; 99.95% for checkout. Browse outage costs revenue but checkout failures cost trust.
4. Inventory consistency: strong; no overselling, especially during Black Friday flash demand.
5. Scale: 200M MAU, 50M DAU, 1.5B page views/day, 5M orders/day average, 5K orders/sec peak.
6. Multi-region: customers in NA, EU, APAC each served from a regional stack with cross-region inventory awareness.

Back-of-the-Envelope Estimation

Traffic

---------- Traffic mix ----------
MAU:                            200M
DAU:                            50M
Page views per DAU:             ~30 (browse heavy)
Total page views/day:           ~1.5B
Product detail QPS avg:         ~17K
Product detail QPS peak:        ~80K
Search QPS avg:                 ~5K
Cart writes/day:                ~50M
Orders/day:                     ~5M (5K/sec at Black Friday peak, 60/sec average)
Read/write ratio:               ~300:1

The 300:1 read/write ratio is the architectural cue: optimize aggressively for reads.

Storage

---------- Storage estimation ----------
Products (100M SKUs * ~10 KB each):    ~1 TB (sharded Postgres)
Product images (100M * 5 photos * 200 KB): ~100 TB (S3 + CDN)
Reviews (1B * ~500 B):                 ~500 GB
Orders (5M/day * 5 yr * ~5 KB):        ~45 TB (sharded by customer)
Cart state (50M active * ~2 KB):       ~100 GB (Redis hot, Postgres cold)
Inventory (100M SKUs * ~32 B):         ~3 GB (Redis hot, Postgres cold)

Bandwidth

---------- Bandwidth ----------
Product images served via CDN:          ~80% offload from origin
Origin egress for images:               ~5 PB/month after CDN
API egress (JSON for product/cart/order): ~50 TB/month

High-Level Design

---------- Service map ----------
[ Client ] -> [ CDN / Edge ] (also serves images/static)
                  |
                  v
          [ API Gateway ] (auth, rate limit)
                  |
   +-----------+--+--+-----------+-----------+
   v           v     v           v           v
 +-------+ +--------+ +--------+ +--------+ +--------+
 |Catalog| | Search | |  Cart  | | Order  | |Reviews |
 +-------+ +--------+ +--------+ +--------+ +--------+
     |         |         |          |          |
     v         v         v          v          v
 +-------+ +--------+ +--------+ +--------+ +--------+
 |Postgrs| |Elastic-| | Redis +| |Postgres| |Postgres|
 |catalog| |search  | |Postgres| |Order DB| |/ Mongo |
 +-------+ +--------+ +--------+ +--------+ +--------+
                                   |
            +----------------------+----------------------+
            v                      v                      v
   +------------------+  +------------------+  +------------------+
   | Inventory Svc    |  | Payment Svc      |  | Fulfillment Svc  |
   +------------------+  +------------------+  +------------------+
            |                      |                      |
            v                      v                      v
   +------------------+  +------------------+  +------------------+
   | Inventory Store  |  | Stripe / PSP     |  | Warehouse        |
   | (Postgres+Redis) |  |                  |  | Routing Service  |
   +------------------+  +------------------+  +------------------+

Key APIs

GET  /api/v1/products/:id                  // product detail
GET  /api/v1/search?q=...&filters=...       // faceted search
GET  /api/v1/cart                           // current customer cart
POST /api/v1/cart/items                     // add or update cart item
POST /api/v1/checkout                       // start checkout (idempotency-key required)
  body: { addressId, paymentMethodId, idempotencyKey }
GET  /api/v1/orders/:id                     // order detail and status
POST /api/v1/orders/:id/return              // initiate return

Read Path (Browse)

1. Client requests product detail.
2. CDN serves images and any cached HTML/JSON shells.
3. API gateway forwards to Catalog Service.
4. Catalog Service hits a Redis cache; on miss, reads Postgres replica and warms cache (TTL 5 min).
5. Inventory snippet (in stock / out of stock) read from Inventory Service Redis store.

Write Path (Checkout)

See the Detailed Design section. The interesting part is the saga.

Detailed Design

The two interesting components are the inventory reservation and the checkout saga.

Inventory Reservation

The naive approach: on order placement, decrement inventory in the database. Problem: between the customer clicking 'Buy' and payment completing, the unit is still 'available' to other customers; two people can both proceed to pay for the last unit.

The correct approach: a TTL hold. When the customer enters checkout, reserve N units for a short window (e.g., 10 minutes). If checkout completes, the hold becomes a permanent decrement; if it expires, units return to the pool.

---------- Inventory state per SKU ----------
on_hand          int    physical units in warehouse
reserved_holds   list   active short-lived reservations [{order_id, qty, expires_at}]
available        view   on_hand - sum(active holds)

The atomic operation is 'reserve N units IFF available >= N', implemented with Postgres row-level locking or a Redis Lua script.

-- Postgres version (per SKU row, pessimistic lock; OK at moderate scale)
BEGIN;
SELECT on_hand, reserved_holds
  FROM inventory
  WHERE sku_id = $1
  FOR UPDATE;                                -- row lock until COMMIT

-- in app: compute available = on_hand - sum(active holds); abort if available < $3
UPDATE inventory
   SET reserved_holds = reserved_holds || jsonb_build_object(
           'order_id', $2, 'qty', $3, 'expires_at', NOW() + interval '10 min'
       )
 WHERE sku_id = $1;
COMMIT;

For flash-sale SKUs (one product, 100K simultaneous reservations), Postgres row locks become a bottleneck. Switch to a Redis Lua script keyed by SKU:

Redis reservation Lua script (KEYS[1] = inventory:sku:<id> hash with fields on_hand and reserved_total; ARGV = qty, order_id, expires_at_ms):

local on_hand = tonumber(redis.call('HGET', KEYS[1], 'on_hand') or '0')
local reserved = tonumber(redis.call('HGET', KEYS[1], 'reserved_total') or '0')
if on_hand - reserved < tonumber(ARGV[1]) then
    return 0
end
redis.call('HINCRBY', KEYS[1], 'reserved_total', tonumber(ARGV[1]))
redis.call('ZADD', KEYS[1] .. ':holds', tonumber(ARGV[3]), ARGV[2] .. ':' .. ARGV[1])
return 1

A background sweeper drops expired holds from the sorted set and decrements reserved_total.

The Checkout Saga

Checkout is a multi-step workflow that touches inventory, payment, and order systems. Distributed transactions across them are too slow and brittle; we use a saga with explicit compensating actions.

---------- Checkout saga ----------
1. validate cart (auth, addresses, totals)
2. reserve inventory                       (compensation: release reservation)
3. charge payment                          (compensation: refund or void)
4. create order record                     (compensation: cancel order)
5. confirm inventory (turn hold into decrement)
6. dispatch fulfillment                    (compensation: cancel shipment)
7. notify customer (email, push)

If any step fails, the saga walks backward and runs the compensating action for each completed step. The orchestrator (Order Service) is the saga coordinator; it persists saga state so a process restart can resume.

// Persisted saga state in the order_sagas table
{
    "saga_id": "sa_abc",
    "order_id": "o_456",
    "customer_id": "u_42",
    "step": "PAYMENT_CHARGED",
    "steps_completed": ["INVENTORY_RESERVED", "PAYMENT_CHARGED"],
    "compensations_to_run": ["REFUND_PAYMENT", "RELEASE_INVENTORY"],
    "updated_at": "2026-04-26T10:00:00Z"
}

Idempotency at Checkout

The client sends an Idempotency-Key header on POST /checkout. The Order Service stores (idempotency_key, response) in a 24-hour cache. A retried checkout with the same key returns the original response without re-reserving inventory or re-charging the card.

async function placeOrder({ cart, addressId, paymentMethodId }) {
    const idempotencyKey = crypto.randomUUID();
    for (let attempt = 0; attempt < 3; attempt++) {
        try {
            const res = await fetch('/api/v1/checkout', {
                method: 'POST',
                headers: { 'Idempotency-Key': idempotencyKey, 'Content-Type': 'application/json' },
                body: JSON.stringify({ cart, addressId, paymentMethodId })
            });
            if (res.ok) return res.json();
            if (res.status >= 500) continue;          // retryable
            throw new Error(await res.text());        // 4xx: do not retry
        } catch (e) { if (attempt === 2) throw e; }
    }
}

Cart Service

Cart is read on every page (mini-cart icon). It must be fast and durable. Two-tier storage:

• Redis holds the live cart (cart:<user_id>) for fast reads / writes.
• Postgres is written asynchronously (every 30 s or on cart update if Redis flush fails) so a Redis crash does not lose carts.

For anonymous users we key on a session cookie; on login we merge the anonymous cart into the user cart.

Catalog and Search

Catalog data lives in Postgres sharded by category or SKU id. Search runs on Elasticsearch with a documented schema per product, indexed asynchronously from the canonical Postgres store via Kafka CDC. Faceted search (price range, brand, color) is what Elasticsearch handles cleanly.

Product images live in S3 and are served via a CDN with multi-resolution variants (thumb-200.jpg, detail-1200.webp).

Recommendations

Personalized recommendations are computed offline (collaborative filtering and content-based signals) and served from a key-value store at request time. We do NOT compute them inline during product detail; that would explode latency.

Data Model

Postgres: products (sharded by sku_id hash)

CREATE TABLE products (
    sku_id          BIGINT PRIMARY KEY,
    title           VARCHAR(512) NOT NULL,
    description     TEXT,
    price_cents     INT NOT NULL,
    currency        CHAR(3) NOT NULL,
    brand_id        BIGINT,
    category_id     BIGINT,
    weight_grams    INT,
    attrs           JSONB,
    is_active       BOOLEAN DEFAULT TRUE,
    created_at      TIMESTAMPTZ NOT NULL,
    updated_at      TIMESTAMPTZ NOT NULL
);

CREATE INDEX idx_products_category ON products (category_id) WHERE is_active;
CREATE INDEX idx_products_brand ON products (brand_id) WHERE is_active;

Postgres: inventory (sharded by sku_id)

CREATE TABLE inventory (
    sku_id              BIGINT PRIMARY KEY,
    warehouse_id        BIGINT,
    on_hand             INT NOT NULL,
    reserved_holds      JSONB NOT NULL DEFAULT '[]',
    version             INT NOT NULL DEFAULT 0,
    updated_at          TIMESTAMPTZ NOT NULL
);

Postgres: orders (sharded by customer_id)

CREATE TABLE orders (
    order_id        BIGINT PRIMARY KEY,
    customer_id     BIGINT NOT NULL,
    status          VARCHAR(32) NOT NULL,        -- PLACED, PAID, SHIPPED, DELIVERED, CANCELLED, RETURNED
    total_cents     INT NOT NULL,
    currency        CHAR(3) NOT NULL,
    shipping_addr   JSONB NOT NULL,
    payment_id      VARCHAR(64),
    placed_at       TIMESTAMPTZ NOT NULL,
    updated_at      TIMESTAMPTZ NOT NULL
);

CREATE TABLE order_items (
    order_id    BIGINT,
    sku_id      BIGINT,
    qty         INT NOT NULL,
    price_cents INT NOT NULL,
    PRIMARY KEY (order_id, sku_id)
);

Redis: cart, inventory hot path, idempotency

---------- Redis layout ----------
cart:<user_id>                 hash {sku_id: qty}                       TTL 30 days
inventory:sku:<sku_id>         hash {on_hand, reserved_total}            no TTL
inventory:sku:<sku_id>:holds   ZSET (member=order_id:qty, score=expires_ms)
idempotency:<key>              JSON serialized response                  TTL 24 h

Elasticsearch: product index

Documents per SKU with text fields (title, description), keyword facets (brand, category, color), numeric fields (price), and a popularity score for ranking.

Scaling and Bottlenecks

Black Friday: 5K orders/sec

• Inventory hot keys: a flash-sale SKU might receive 50K reservation attempts/sec for 100 units. Use the Redis Lua reservation; the script is single-threaded per shard but runs in microseconds. Once available - reserved == 0, subsequent calls return 0 immediately.
• Checkout queue: if the Order Service is overloaded, queue checkouts in Kafka and process at capacity; show the customer a 'placing your order' state until the saga completes.
• Payment provider rate limits: Stripe enforces its own rate limit; we partition our API key usage and use idempotency keys so that retries during throttling are safe.

Black Friday: 80K product-detail QPS

• Edge cache product detail responses for 60 s. Even at 80K QPS, 95% hits at the edge means origin sees 4K QPS, well within Postgres replica capacity.
• Inventory snippet on the product page is more dynamic; fetch it via a side request to Inventory Service that returns in stock / out of stock with a 5 s TTL. We deliberately split the cache TTLs: the rest of the product detail (title, description, images) tolerates a 60 s lag, but the in-stock badge would be misleading if it lagged 60 s during a flash sale, so it bypasses the long edge cache.

Multi-Warehouse Routing

When a customer in Seattle orders a Kindle, the order may fulfill from one of several warehouses. Routing service picks based on stock availability + distance + shipping speed promised. If the closest warehouse runs out mid-order, the routing service splits the shipment across two warehouses.

---------- Routing decision ----------
Preferred warehouse:    Sumner, WA       (closest, 1-day Prime promise)
Fallback 1:             Reno, NV         (2-day)
Fallback 2:             Phoenix, AZ      (2-day)
If Sumner is out:       split across Reno + Phoenix

Catalog Index Lag

Elasticsearch indexing from CDC has 10-30 s lag. After a price change, the catalog API shows the new price within 1 s (Postgres replica) but search may return the old price for ~30 s. Acceptable; we display the canonical price on the product detail page, where the click lands.

Returns and Refunds

Return flow is a separate saga (RMA): customer requests, label issued, package received at warehouse, inspection passes, refund issued via payment provider. Failure modes: package never arrives, inspection fails. Each has a compensating path or escalation to ops.

Fraud and Risk

A risk service screens checkout in under 200 ms (synchronously) using model scores plus deterministic rules. High-risk orders enter a manual review queue rather than immediately charging; low-risk orders proceed.

Trade-offs and Alternatives

Saga vs Two-Phase Commit

A distributed 2PC across inventory, payment, and order would block all participants for the duration of the transaction; under load this is unworkable. A saga gives up atomicity (each step commits independently) in exchange for availability and clear failure handling via compensations. The cost is reasoning about partial-failure states; the saga state machine in a database makes this tractable.

Pessimistic vs Optimistic Locking for Inventory

Postgres row locks are simple and correct for moderate contention. They become a bottleneck for flash-sale SKUs (100K simultaneous reservations on one row). Optimistic locking (version + retry) plus the Redis fast path handles flash demand without serializing on a database row.

Strong vs Eventual Consistency for Inventory

We choose strong inventory consistency. Overselling damages trust and creates expensive customer service work; the latency cost (one Redis hop + Postgres write inside checkout) is acceptable. Search-index lag is eventually consistent because seeing slightly stale prices in search is harmless.

Single Postgres vs Sharded

At 100M SKUs, a single Postgres can serve catalog reads from replicas but writes (price updates, inventory changes) bottleneck. Shard by sku_id hash for products and inventory; shard by customer_id for orders. Choose shard keys per access pattern: customers query their own orders, so customer_id is the natural shard key.

Synchronous vs Asynchronous Fulfillment

Checkout returns success after payment + order creation (synchronous). Fulfillment dispatch (warehouse pick, ship) is asynchronous via a Kafka topic; the warehouse system processes events at its own pace. This decouples the checkout latency from the warehouse system's throughput.

Why Idempotency Keys, Not Server-Generated Order IDs?

The client knows when it intends one checkout; the server only sees a request. If the network drops the response, the client retries and the server cannot tell it is a retry without help. The idempotency key (client-generated UUID) lets the server recognize and replay the original response. Server-generated order IDs are still used internally; the idempotency key is the external retry-safe handle.