System Design: News Feed 2026 [Twitter/Instagram Architecture]

Last Updated: June 2026

Why News Feed Design is a Benchmark Problem

Candidates report news feed design in roughly 15% of FAANG system design rounds and frequently at social product companies. Based on public preparation resources and candidate-reported interview threads, it is one of the best problems for testing your knowledge of fan-out tradeoffs, caching strategies, and handling heterogeneous user types (regular vs celebrity accounts).

Step 1: Requirements

Functional requirements:

• Users can create posts (text, image, video)
• Users can follow/unfollow other users
• Feed shows recent posts from people the user follows, newest first
• Feed supports pagination (infinite scroll)
• Like and comment on posts

Non-functional requirements:

• Scale: 500M users, 1B daily active sessions
• Post creation: roughly 1M posts/hour = ~278 posts/sec
• Feed reads: roughly 10B feed reads/day = ~115K/sec
• Read-heavy: 100:1 read-to-write ratio
• Eventual consistency for feed: 1-2 second delivery lag acceptable

Step 2: Capacity Estimation

Posts per day: ~24M (1M/hour)
Average followers per user: 200
Fan-out writes: 24M * 200 = 4.8B feed writes/day = ~55K feed writes/sec

Feed storage per user:
  Keep last 1000 posts in feed cache
  Each post reference: ~32 bytes (post_id + timestamp)
  Per user: 1000 * 32 = 32KB
  For 100M active users: 3.2TB feed cache (needs Redis cluster)

Post storage:
  Text: 100M posts * 200 bytes avg = 20GB/day
  Images/Video: stored in S3, URL in post table

Step 3: API Design

Define the public API before the internals. Three endpoints carry almost all the traffic.

GET /v1/feed?cursor={post_id}&limit=20
  Returns the authenticated user's feed page.
  Cursor-based pagination (not offset) because offset pagination breaks
  when new posts arrive between page loads. The cursor is the post_id of
  the last item from the previous page.
  Response: { posts: [...], next_cursor: "987654321" }

POST /v1/posts
  Body: { content, media_ids: [...], post_type }
  Creates a post and triggers fan-out. Returns the created post_id
  (a Snowflake ID, time-sortable) immediately; fan-out happens async.

POST /v1/follow
  Body: { followee_id }
  Adds an edge to the follow graph. If the followee is a celebrity,
  the new follower is NOT back-filled with old celebrity posts; they
  appear on the next read-time merge.

Cursor pagination is the single most common API mistake candidates make on this problem. Offset pagination (?page=3) double-counts or skips posts because the underlying list shifts as new posts arrive. Always use a stable cursor keyed on the time-sortable post_id.

Step 3b: Core Architecture

[Client]
   |
[API Gateway]
   |
[Feed Service] <-----> [Feed Cache (Redis)]
   |
[Post Service] <-----> [Post DB (MySQL)]
   |
[Fan-out Service] <-----> [Message Queue (Kafka)]
   |
[User Graph Service] <-> [Graph DB or Adjacency List in MySQL]
   |
[Media Service] <------> [S3 + CDN]

Step 4: Fan-out Strategies

Fan-out on Write (Push Model)

When user A posts, the fan-out service looks up all of A's followers and pushes the post_id to each follower's feed cache immediately.

class FanoutService:
    def __init__(self, user_graph, feed_cache, kafka):
        self.graph = user_graph
        self.cache = feed_cache
        self.kafka = kafka

    def handle_new_post(self, post_id, author_id):
        followers = self.graph.get_followers(author_id)

        # Batch write to Kafka, workers fan out to Redis
        self.kafka.produce('fanout', {
            'post_id': post_id,
            'author_id': author_id,
            'follower_ids': followers,
            'timestamp': time.time()
        })

class FanoutWorker:
    def process(self, event):
        for follower_id in event['follower_ids']:
            # Add to follower's feed sorted set
            self.cache.zadd(
                f"feed:{follower_id}",
                {event['post_id']: event['timestamp']}
            )
            # Trim to last 1000 posts
            self.cache.zremrangebyrank(f"feed:{follower_id}", 0, -1001)

Pros: Reads are fast (just read from cache). Cons: Write amplification. A user with 1M followers creates 1M fan-out writes per post.

Fan-out on Read (Pull Model)

When user B requests their feed, the system fetches the latest posts from all accounts B follows in real time.

def get_feed_pull(user_id, limit=20):
    followees = user_graph.get_followees(user_id)

    # Fetch latest post_id from each followee
    all_posts = []
    for followee_id in followees:
        posts = post_service.get_latest_posts(followee_id, limit=3)
        all_posts.extend(posts)

    # Sort by timestamp, return top limit
    all_posts.sort(key=lambda p: p.timestamp, reverse=True)
    return all_posts[:limit]

Pros: No write amplification. Cons: Each feed load requires N DB reads (N = number of followees). Very slow for users following thousands of accounts.

Hybrid Approach (Production Standard)

Regular users (< 10K followers): fan-out on write
  - Their posts written to followers' feed caches on post creation
  - Feed reads are fast: just read from own cache

Celebrity users (> 10K followers): fan-out on read
  - Their posts stored in their own timeline, not fanned out
  - Feed service identifies which followees are celebrities
  - Merges pre-built feed cache with real-time celebrity posts

Feed generation:
  base_feed = redis.zrange(f"feed:{user_id}", 0, 19)  # pre-built, O(log n)
  celebrity_posts = fetch_celebrity_posts(user.celebrity_followees)
  merged = merge_sorted(base_feed, celebrity_posts)[:20]

Step 5: Database Schema

-- Posts table
CREATE TABLE posts (
    post_id     BIGINT PRIMARY KEY,    -- Snowflake ID (time-sortable)
    user_id     BIGINT NOT NULL,
    content     TEXT,
    media_url   VARCHAR(2048),
    post_type   VARCHAR(20),           -- text, image, video
    like_count  INT DEFAULT 0,
    created_at  TIMESTAMP DEFAULT NOW(),
    INDEX (user_id, created_at DESC)
);

-- User follows (adjacency list)
CREATE TABLE follows (
    follower_id BIGINT,
    followee_id BIGINT,
    created_at  TIMESTAMP DEFAULT NOW(),
    PRIMARY KEY (follower_id, followee_id),
    INDEX (followee_id)                -- for "who follows me" queries
);

-- Feed cache (in Redis)
-- ZADD feed:{user_id} {timestamp} {post_id}
-- ZRANGE feed:{user_id} 0 19 WITHSCORES REVRANGEBYSCORE

Step 6: Feed Ranking

A pure chronological feed was the original Twitter design. Modern feeds use ML ranking to surface more relevant posts. The ranking signal includes:

• Recency (decay function over time)
• Engagement signals (likes, comments, shares from your network)
• Relationship strength (how often you interact with the author)
• Content type preference (if you engage more with videos)

The feed service first retrieves a candidate set (roughly 200 posts) and the ranking model scores them. The ranked top 20 are returned. This pipeline adds latency but significantly increases engagement.

In a system design interview, you should mention that the ranking system is a separate service that the feed service calls. You do not need to design the ML model itself.

Step 7: Caching Strategy

Layer 1: CDN (edge cache)
  - Cache feed responses for 30 seconds
  - Served from closest geographic node
  - Reduces origin server load by roughly 70%

Layer 2: Application cache (Redis)
  - Feed cache per user: sorted set of post_ids
  - Post content cache: hash of post_id -> post data
  - TTL: 24 hours; refresh on each feed request

Layer 3: Database (MySQL)
  - Source of truth
  - Direct queries only for cache misses

Step 8: Handling Edge Cases

User with many followees (following 10K accounts): Even with hybrid fan-out, merging 10K celebrity feeds at read time is slow. Cap celebrity list at 500 for real-time merge; rest are pulled lazily on scroll.

Post deletion: Set a is_deleted flag. Cache entries for deleted posts are filtered on read. Background job cleans up feed caches asynchronously.

New user (cold start): No followers yet, feed is empty. Show trending posts, suggested accounts to follow, and posts from interest categories to bootstrap the experience.

Deep Dive: The Celebrity Merge in Detail

The hybrid model's hard part is the read-time merge. When a user opens their feed, the feed service does the following:

def get_feed(user_id, cursor=None, limit=20):
    # 1. Read the pre-built feed (posts from regular followees, pushed on write)
    base = redis.zrevrange(f"feed:{user_id}", 0, 200, withscores=True)

    # 2. Identify which followees are celebrities (cached per user)
    celeb_ids = redis.smembers(f"celeb_followees:{user_id}")

    # 3. Pull recent posts from each celebrity's own timeline
    celeb_posts = []
    for cid in celeb_ids:
        celeb_posts += redis.zrevrange(f"timeline:{cid}", 0, 20, withscores=True)

    # 4. Merge two sorted streams by timestamp (k-way merge)
    merged = heapq.merge(base, celeb_posts, key=lambda x: x[1], reverse=True)

    # 5. Hydrate post_ids into full post objects (batch get from post cache)
    page = take_after_cursor(merged, cursor, limit)
    return hydrate_posts(page)

The merge is a k-way merge of sorted streams, which is O((B + C) log K) where B is base feed size, C is total celebrity posts, and K is the number of celebrity timelines. Because each stream is already sorted by timestamp in the Redis sorted set, the merge is cheap. The expensive part is step 5, hydration, which is why a batch MGET against the post cache (one round trip for all post_ids) matters more than the merge itself.

A subtle correctness point: the celebrity timeline must be capped (step 3 reads only the latest 20 per celebrity) because a celebrity who posts hundreds of times a day could otherwise flood the merge. Capping per-source keeps any single account from dominating the feed.

Step 9: Failure Handling

Fan-out worker crashes mid-fan-out:
  Kafka consumer offset is committed only after the full follower batch
  is written. On restart, the worker reprocesses the batch. zadd is
  idempotent (same post_id + score), so duplicate processing is safe.

Redis feed cache node fails:
  Feed cache is a derived view, not source of truth. On a cache miss or
  node loss, rebuild the user's feed from the post DB by querying their
  followees' recent posts. The post DB is the durable store.

Fan-out lag during a traffic spike:
  Fan-out is async via Kafka, so a spike grows the queue rather than
  failing writes. Feed delivery lag rises from 1-2s to maybe 10s during
  the spike, which is acceptable per the non-functional requirements.
  Auto-scale fan-out workers on consumer lag.

The key insight to state in the interview: the feed cache is a materialized view that can always be rebuilt from the durable post store, so losing it degrades latency but never loses data.

Follow-up Questions Interviewers Ask

How do you guarantee a user never sees a post from someone they unfollowed? Unfollow removes the graph edge but old posts may still sit in the pushed feed cache. Filter at read time against the current followee set, or run an async cleanup that removes the unfollowed author's posts from the cache. Read-time filtering is simpler and authoritative.

How do you shard the post database? Shard by user_id (the author). All of a user's posts live on one shard, so fetching a user's timeline is a single-shard query. Snowflake post_ids embed a timestamp, so within a shard, posts are already time-ordered without a separate sort.

How do you support feed for a user across multiple devices? The feed lives server-side keyed by user_id, not per device. Each device reads the same feed with its own cursor position, stored client-side. The server is stateless with respect to device.

How does the read-time merge stay fast for a user following 5,000 accounts? Most of those 5,000 are regular users whose posts are already in the pushed feed cache. Only the celebrity subset is merged at read time, and that subset is capped (500 max real-time, rest pulled lazily on scroll). So merge cost scales with celebrity count, not total followees.

Tradeoffs Summary

Related Articles

• System Design Interview Questions 2026
• System Design Chat Application 2026
• System Design Notification Service 2026
• System Design Rate Limiter 2026
• Must-Do Coding Questions