What is a Cache?

A cache is a smaller, faster copy of data placed close to whoever asks for it, so the next request can skip the expensive trip to the source of truth.

Three numbers explain why caches exist at all (Jeff Dean's classic latency table, rounded for memory):

Each jump is 100 to 1000 times slower than the previous one. Caching is the act of remembering data at one level so future reads do not have to drop down to the next.

Hit, miss, eviction, TTL

Four words you must know cold:

• Cache hit: the requested key is in the cache; the slow source is not touched.
• Cache miss: the key is absent; the system must fetch from the source and (usually) populate the cache.
• Eviction: a key is removed to make room for newer ones (driven by an eviction policy).
• TTL (time-to-live): an expiration timestamp. Once it passes, the entry is treated as a miss even if it is still in memory.

The single most important metric for any cache is hit rate: hits divided by (hits + misses). A 95% hit rate means only 5 of every 100 requests reach the database. A 50% hit rate means your cache is barely helping and may even be hurting (extra hop + double the writes).

How a Cache Sits in a Request

A typical read against a cache-aside cache (the most common pattern):

---------- Cache-aside read flow ----------
   client                                                      
     |                                                         
     v                                                         
  [ app server ] - 1. GET key --> [ cache (Redis) ]           
                  <-- 2a. hit ----- (return value, done)       
                  <-- 2b. miss ----                            
                                                               
  [ app server ] - 3. SELECT --> [ database (Postgres) ]      
                  <-- 4. row -----                             
  [ app server ] - 5. SET key --> [ cache (Redis) ]           
     |                                                         
     v                                                         
  return to client

Notice the two important details:

1. The application code (not the cache) decides when to read from and write to the cache.
2. On a miss, the application is responsible for populating the cache.

Where Caches Live

A real system stacks several caches, each closer to the user than the last. Understanding the stack matters because each layer has different invalidation, capacity, and consistency rules.

---------- Cache hierarchy ----------
   user device
     | 
     v
  [ browser cache ]      ETag / Cache-Control headers, ~hundreds of MB
     |
     v
  [ CDN edge cache ]     Cloudflare / Fastly / CloudFront, ~hundreds of GB per POP
     |
     v
  [ reverse proxy cache ] NGINX / Varnish, in your datacenter
     |
     v
  [ application cache ]  in-process LRU map, microsecond access
     |
     v
  [ remote cache ]       Redis / Memcached cluster, single-digit ms
     |
     v
  [ database cache ]     Postgres shared_buffers, MySQL InnoDB buffer pool
     |
     v
  [ disk / source of truth ]

A request that misses every layer pays the full cost. The art is to make as few requests as possible reach the bottom.

Read Patterns

There are two ways application code can interact with a cache for reads.

1. Cache-Aside (Lazy Loading)

The application is in charge. It checks the cache first; on a miss, it reads from the database and writes the result back into the cache.

async function getUser(userId) {
    const key = `user:${userId}`;
    let user = await redis.get(key);
    if (user) return JSON.parse(user);

    user = await db.query('SELECT * FROM users WHERE id = $1', [userId]);
    if (user) {
        await redis.set(key, JSON.stringify(user), 'EX', 300); // 5 min TTL
    }
    return user;
}

Pros: simple, only caches data that is actually requested, resilient to cache failures (a down cache means slower reads, not broken reads). Cons: each new key takes a full database round trip on first access (cold cache problem); the application carries the caching logic.

This is the default pattern for most web apps and the pattern Memcached and Redis are usually used with.

2. Read-Through

The cache is in charge. The application asks the cache for a key; if missing, the cache itself fetches from the database and stores the result. The application never talks to the database directly for cached data.

---------- Read-through ----------
  app - GET key --> [ cache library ]
                       | (on miss)
                       v
                     [ database ]
                       |
                       v
  app <-- value ---- [ cache library ] (now populated)

Pros: application code is clean, no boilerplate cache lookups. Cons: requires a cache library that knows how to talk to your database (Hibernate L2, AWS DAX for DynamoDB, Apollo Client for GraphQL). A cache outage is also a database-access outage because the app does not know how to bypass it.

Write Patterns

The interesting design decisions live on the write side. There are three names every interviewer expects you to know.

Write-Through

Every write goes to both the cache and the database synchronously. The write is acknowledged only after both succeed.

---------- Write-through ----------
  client - write --> [ cache ] - write --> [ database ]
                          |                       |
                          v                       v
                       (both updated; ack only after both succeed)

Pros: the cache is always consistent with the database. Subsequent reads always hit fresh data. Cons: writes are slower because they pay two round trips. Unused data is cached unnecessarily (every write populates the cache, even for keys that will never be read again).

Use when: read-heavy workload over data that is updated occasionally and read often. Examples: user profile fields, product catalog, configuration.

Write-Back (also called Write-Behind)

The write goes to the cache only; the cache acknowledges immediately and asynchronously flushes the change to the database in the background.

---------- Write-back ----------
  client - write --> [ cache ] (ack immediately)
                          | (async batch flush)
                          v
                      [ database ]

Pros: very fast writes; the cache batches updates so the database sees fewer, larger writes. Great for write-heavy workloads (analytics counters, view counts, leaderboards). Cons: data loss risk - if the cache crashes before the flush, recent writes are gone. The cache is now part of your durability story; you need replication or a write-ahead log on the cache itself.

Use when: high write throughput, some data loss is acceptable, or the cache is durable (Redis with AOF persistence, for example).

Write-Around

Writes skip the cache and go straight to the database. The cache is populated only when a subsequent read misses.

---------- Write-around ----------
  client - write --> [ database ]
                          (cache untouched on write)
  client - read  --> [ cache ] (miss)
                          |
                          v
                      [ database ] - value --> populate cache

Pros: avoids polluting the cache with write-once data (logs, audit events, sensor readings). Cons: the write-then-immediate-read pattern always misses the cache. May be combined with a short TTL to avoid serving an old cached copy.

Use when: write-heavy data that is rarely re-read. Examples: append-only logs, audit trails, IoT sensor batches.

Decision matrix

Eviction Policies

A cache has finite memory. When it fills up, it must evict something to make room. The choice of policy directly drives hit rate.

• LRU (Least Recently Used): evict the entry that was accessed longest ago. The default in Redis (allkeys-lru), Memcached, OS page caches, browsers. Works well for skewed workloads where some keys are read repeatedly.
• LFU (Least Frequently Used): evict the entry with the fewest accesses. Better than LRU for keys with seasonal popularity (e.g., a flash sale item that should stay cached even if it was just read minutes ago). Available as allkeys-lfu in Redis 4+.
• FIFO (First In, First Out): evict the oldest insertion regardless of recent access. Simple but rarely the best fit; mostly seen in queue-like caches.
• TTL-based (expiration): not strictly an eviction policy but combined with the above. Redis EXPIRE key 300 says 'this entry self-deletes after 5 minutes'.
• Random: evict a random key. Surprisingly competitive when paired with TTL; cheap to implement.

Rule of thumb: start with LRU + a TTL. Move to LFU only if profiling shows certain keys keep getting evicted before their next access.

Failure Modes (the part interviewers love)

Cache Stampede (Thundering Herd)

A popular cache entry expires. A thousand requests arrive in the same second, all miss, all hammer the database with the same query. The database falls over, the cache stays empty, the system collapses.

---------- Cache stampede ----------
  T0   key 'top-products' expires in cache
  T0   1000 concurrent requests arrive
  T0   1000 misses --> 1000 SELECTs against the same row
  T0+1 database overloaded, all requests time out

Mitigations (use one or more):

1. Locking / single-flight: only one request is allowed to recompute the value; the rest wait for it. Built into Go's singleflight, easy in Redis with SETNX.
2. Stale-while-revalidate: serve the expired value to most callers while one background worker refreshes it. Used by browsers, NGINX, and the stale-while-revalidate HTTP header.
3. Probabilistic early expiration: each requester refreshes the value with small probability before the TTL expires, so refreshes are spread out instead of synchronized.
4. Pre-warm: refresh the entry before it expires (a cron job that recomputes the homepage every minute).

Hot Keys

One key receives a huge fraction of traffic (the homepage of a viral video, the product page of a flash sale). A single cache shard becomes a bottleneck.

Mitigations:

• Replicate the hot key across multiple cache nodes; clients pick a replica with a consistent hash on a salt.
• Local in-process cache in front of the remote cache for the top-N keys, with a short TTL (10 to 60 seconds).
• Read-through CDN for content that can be served from the edge.

Stale Reads

With write-back and TTL-based caches, the cache lags the source. A user updates their profile picture and refreshes; the old picture appears for the next 5 minutes.

Mitigations:

• Invalidate on write: after updating the database, delete the key from the cache (redis.del('user:42')). On the next read, the cache repopulates.
• Write-through for user-visible mutable data; trade write latency for freshness.
• Versioned keys: include a version or timestamp in the key (user:42:v17); writing increments the version, old keys age out via TTL.