Fundamentals

Functional vs non-functional

Functional requirements are what the system does — "users can post photos", "followers see them in a feed". Non-functional requirements are the properties that shape the architecture — latency, availability, consistency, durability, cost. Functional requirements determine your API; non-functional requirements determine your architecture. Interviewers mostly grade the second.

The clarifying questions that actually change the design

Ask five, not fifteen. These five change the answer more than any others:

Back-of-envelope arithmetic

Round aggressively. The interviewer wants the order of magnitude and the reasoning, not the digits.

# The one conversion to memorise
1,000,000 / day  ≈  12 / second        # 86,400s/day, round to 100k
1 / second       ≈  86,400 / day       # ~2.6M/month

# Peak is never average
peak QPS = average QPS × 2 to 10       # use ×3 unless told otherwise

# Storage
bytes/day = writes/day × bytes/write
5 years   = bytes/day × 365 × 5 × replication_factor (usually 3)

# Bandwidth
bandwidth = QPS × payload_size         # do reads and writes separately

# Fleet size
servers = peak QPS / per-server QPS    # then × 1.5 for headroom + failures

Worked example — photo sharing, 100M DAU

Users                100M DAU
Photos posted        1 per user per 10 days  →  10M photos/day
Feed views           20 per user per day     →  2B reads/day

WRITES   10M/day        ≈  116 /s   →  peak ×3  ≈  350 /s
READS    2B/day         ≈  23,000 /s → peak ×3  ≈  70,000 /s
Ratio    ~200:1 read-heavy  →  cache + read replicas, obviously

STORAGE  photo 2 MB + thumbnails 0.5 MB  ≈  2.5 MB
         10M × 2.5 MB   =  25 TB/day
         × 365 × 5      ≈  45 PB over 5 years  (before replication)
         metadata: 10M × 1 KB = 10 GB/day → 18 TB in 5y  ← trivially small

BANDWIDTH  egress  70,000 /s × 300 KB (a feed page of thumbs)  ≈  21 GB/s
           → this is a CDN problem, not an origin problem

Conclusion in one line: metadata is small enough for a sharded SQL cluster;
blobs go to object storage behind a CDN; the read path is entirely cache-shaped.

Latency: budget it, and always at a percentile

A 200 ms p99 target is a budget you spend across hops. Write it down and each component has to fit:

Where candidates lose points

• Estimating for a scale nobody asked for. If they said 1M users, designing for 1B is not ambition, it's not listening.
• Quoting an average and forgetting peak. Traffic is diurnal and spiky.
• Forgetting replication and indexes when sizing storage — 3× replication plus indexes can double or triple the raw number.
• Averaging latency. "Average 50 ms" hides a p99 of 4 seconds, and the p99 is what your users feel.

Choosing a protocol

Pagination — offset vs cursor

This is the single most common API design question, and the answer is almost always cursor.

The cursor should be an opaque, encoded token — base64 of {last_sort_key, last_id} — not a raw offset. Opaque means you can change the underlying sort later without breaking old clients. Always tie-break on a unique column: sorting by created_at alone silently drops or repeats rows when timestamps collide.

Idempotency at the API layer

Any unsafe operation a client might retry needs an idempotency key. The client generates a UUID per logical intent and sends it as a header; the server stores the key with the response and replays that response on a repeat.

POST /v1/payments
Idempotency-Key: 8f14e45f-ea6c-4a1e-9c5b-2b31a1f7d0aa
{ "amount": 4999, "currency": "INR", "source": "card_9x2" }

# Server logic
1. INSERT key with state=IN_PROGRESS   # unique constraint = the lock
   - conflict + IN_PROGRESS  → 409 (client retries later)
   - conflict + COMPLETED    → return the stored response, do nothing
2. do the work
3. store the response body + status against the key, state=COMPLETED
4. expire the key after 24h

Note the key is stored with the response, not just as a "seen" flag — a retry must return the same body, or the client can't tell success from a new failure. See topic 14 for the deeper treatment.

Verbs, status codes, and errors

Codes worth getting right: 400 malformed, 401 not authenticated, 403 authenticated but not allowed, 404 absent (also use it to hide existence from unauthorised callers), 409 conflict/version mismatch, 422 semantically invalid, 429 rate limited (always with Retry-After), 503 overloaded — retry, 504 upstream timeout — retry carefully.

Return machine-readable errors: a stable code string clients can branch on, a human message, and a request_id for support. Never make clients regex your prose.

Versioning and compatibility

• URL versioning (/v1/) — ugly, obvious, and it works. Best default for public APIs.
• Header versioning — cleaner URLs, but easy for clients to omit and hard to see in logs and caches.
• Additive-only evolution — the real answer for internal services. Add fields, never remove or repurpose them; make new fields optional with a sensible default. Protobuf enforces this by design if you never reuse field numbers.

Rule of thumb: adding an optional field is safe; removing a field, renaming one, tightening validation, or changing a default is a breaking change even if the type is unchanged.

L4 vs L7

Algorithms

Health checks

• Active — the balancer polls /healthz on an interval. Cheap and predictable. Use hysteresis: 3 consecutive failures to eject, 2 successes to return, so one blip doesn't flap the fleet.
• Passive — infer health from real traffic (5xx rate, timeouts) and eject outliers. Catches failures a synthetic probe misses.
• Shallow vs deep — a shallow check proves the process is up. A deep check pings the database too. Deep checks are a trap: when the shared database hiccups, every server fails its check simultaneously and the balancer removes the entire fleet, converting a degradation into an outage. Keep the load-balancer check shallow; monitor dependencies separately.
• Connection draining — on deploy or scale-in, stop sending new requests but let in-flight ones finish (typically 30 s). Without it, every deploy is a small burst of 502s.

Sticky sessions

Stickiness pins a client to one server, usually via a cookie. It exists because someone put session state in server memory. It costs you: uneven load, a full session loss when that server dies, no clean scale-in, and harder deploys.

The better answer is almost always to remove the need for it — put session state in Redis or a signed JWT and make every server interchangeable. Legitimate remaining uses: in-memory upload buffers, long-lived WebSocket connections (which are inherently sticky), and local caches you want to keep warm (use consistent hashing rather than cookies for that).

Above the load balancer

• DNS round robin — cheap multi-region spreading, but clients and resolvers cache TTLs and ignore failures, so failover is slow and partial.
• GeoDNS / latency-based routing — return the nearest healthy region's IP. Common front door for multi-region.
• Anycast — one IP announced from many locations; the network routes to the closest. Instant failover, no DNS TTL problem. This is how CDNs and public DNS resolvers work.

The layers, from cheapest to most expensive

Patterns

Eviction policies

Redis specifics worth naming: allkeys-lru for a pure cache, volatile-lru when the same instance also holds non-expiring data, and noeviction (write errors when full) when losing keys is unacceptable — which usually means it wasn't a cache.

The four failure modes, and their fixes

Sizing and measuring

Hit rate needed to protect the DB:
  DB QPS = total QPS × (1 − hit rate)
  70,000 QPS at 95% hit  →  3,500 QPS to the DB     # fine
  70,000 QPS at 80% hit  →  14,000 QPS to the DB    # not fine
  # Going 95% → 99% cuts DB load 5×. The last few points matter most.

Memory:
  hot set = items × avg size × (1 + overhead ~0.2)
  1M sessions × 2 KB × 1.2  ≈  2.4 GB   # one node, comfortably

Watch, in order: hit rate, p99 latency, eviction rate, memory %.
A rising eviction rate is the leading indicator — it precedes the hit-rate drop.