Lesson 6.3 · 25 min
News feed
Move the push/pull threshold across a heavy-tailed follower graph and watch writes trade for reads, then hydrate a page in 3 ms instead of 81 and see why offsets duplicate posts.
The idea in plain words
A news feed shows you the latest posts from everyone you follow. The design question is when to do the work: copy each new post into every follower’s feed as it is posted, or gather posts when someone opens their feed.
The problem it solves
Opening the app should show recent posts from everyone you follow, instantly. Building that list from scratch on every open is too slow, and copying every post to every follower is too expensive for accounts with millions of followers.
How it works
- When someone posts, store the post once.
- Push: add the post’s id to each follower’s precomputed feed list (fast reads, more work per post).
- Pull: when someone opens the app, fetch recent posts from the people they follow and merge them (cheap posts, slower reads).
- Most systems mix both: push for normal accounts, pull for celebrities.
- The feed holds ids only; post details are loaded in batches from a cache.
Example
A user with 200 followers posts: 200 small writes, done in milliseconds. A celebrity with 50 million followers posts: pushing would mean 50 million writes, so instead their posts are pulled and merged in when fans open their feed.
Key terms
- Fan-out
- Delivering one post to many followers’ feeds.
- Push (fan-out on write)
- Add the post to each follower’s feed when it is posted; reads are fast.
- Pull (fan-out on read)
- Fetch posts from everyone you follow when you open the feed; writes are cheap.
- Celebrity problem
- An account with millions of followers makes push too expensive.
- Cursor pagination
- Loading the next page from “after this post” rather than “page 5”, so new posts don’t shift results.
In this lesson you'll learn
- How to size a feed: posts, fan-out writes, feed loads and cache memory.
- Push vs pull vs hybrid fan-out on a heavy-tailed follower graph, and where the threshold goes.
- How to hydrate and paginate a feed, and how to defend the whole design.
Sections
- Requirements and numbers
- API and data
- Architecture
- Deep dive: fan-out
- Deep dive: read path
- Design review
Cheat sheet
| Feed | capped list of post ids |
|---|---|
| Fan-out | push most, pull big accounts |
| Inactive users | skip, rebuild on login |
| Hydrate | parallel multi-gets |
| Paginate | cursor, never offset |
The interactive simulations and the scored design review load with JavaScript.