Lesson 4.1 · 20 min
Caching and CDNs
Race five eviction policies on the same 20,000 reads, replay the stale-cache race, defuse a stampede on a hot key, and tune a CDN cache key.
The idea in plain words
A cache is like keeping the things you use most on your desk instead of in the basement. It is fast, but small, so you must decide what to keep there and how long a copy can be trusted before it goes stale.
The problem it solves
Some data is read far more often than it changes, and fetching it from the database every time is slow and puts load where it is most expensive.
How it works
- On a read, look in the cache (fast memory) first.
- Hit: return it. Miss: read the database, store the result in the cache, then return it.
- Give entries a time to live so they eventually refresh.
- When the data changes, delete the cached copy so the next read gets the new value.
- When the cache is full, throw out entries that are least likely to be needed again.
Example
A product page takes 10 ms from the database and 1 ms from cache. With 95% of reads hitting the cache, the average is about 1.5 ms and the database sees only 1 read in 20, so it can serve 20× the traffic.
Key terms
- Cache hit / miss
- A hit finds the answer in the cache; a miss has to go to the slow source.
- Hit ratio
- The share of reads the cache answers, such as 95%.
- Eviction
- Throwing an entry out when the cache is full (for example the least recently used).
- TTL
- Time to live: how long an entry is kept before it expires.
- Stale data
- A cached copy that no longer matches the database.
- CDN
- A cache for files, spread across servers near users.
In this lesson you'll learn
- How LRU, LFU, FIFO and random eviction compare on the same traffic, and what hit ratio buys in latency.
- Cache-aside, write-through and write-behind, and the race that leaves stale data behind.
- How to stop a stampede on a hot key, and how CDN cache keys make or break the hit ratio.
Sections
- Theory: eviction
- Theory: writes and staleness
- Theory: stampedes
- Theory: CDN cache keys
- Design: pick a strategy
- Check yourself
Cheat sheet
| Avg read | 1 ms + miss ratio × DB time |
|---|---|
| LRU / LFU | scans hurt LRU; LFU needs aging |
| On write | update the DB, then delete the key |
| Stampede | single-flight or stale-while-revalidate |
| CDN statics | hashed names, max-age 1 year |
The interactive simulations and the quiz load with JavaScript.