AI System Design
← Learn
Level 2beginner

Caching

Your database is dying. Add a cache.

Depth:
1

Mission

Reads are hammering your database far beyond what it can serve. Most requests ask for the same popular data over and over. Save the database.
2

Interactive Simulation

Before we explain anything — play. Push it until it breaks, then fix it.

Your database is dying — 10K/s on a 2.0K/s DB. Add a cache.
Requests
10K/s
Latency
6.00s
p95 18.01s
Error rate
80.0%
CPU
100%
Cache hit
0.0%
DB load
100%
Est. cost
$660/mo
illustrative
Accepted
2.0K/s
Rejected
8.0K/s
Latency (ms)
Error rate (%)

Load

Cache

Break it

A hot key expires and every request stampedes the DB at once.

System Score18
3

What just happened?

A cache in front of the database served the hot data from memory, so only cache misses reached the DB. Database load collapsed and latency dropped, because memory is far faster than disk.

4

The concept

A cache stores frequently accessed data in fast memory so repeat reads skip the slow backend. Key knobs: cache size (how much of the working set fits), TTL (how long entries stay fresh), and eviction policy (LRU/LFU/FIFO — what to drop when full). Hit rate is the fraction of requests served from cache; higher hit rate means less DB load.

5

Trade-offs

Nothing is free. Here's what this solution costs you.

Staleness
Cached data can be out of date until TTL expiry or invalidation.
Stampede risk
Expiring a hot key can flood the DB with simultaneous misses.
Memory cost
Bigger caches cost more; you rarely cache 100% of data.
6

In the real world

Conceptually similar to a Redis/Memcached tier in front of a relational database.

7

Mini quiz

Question 1 of 40 correct

Cache hit rate goes UP when you…

8

Interview me

The app becomes your interviewer. One question, in your own words.

9

Boss challenge

Survive a hot-key stampede

10,000 req/s on a 2,000 req/s DB, and a hot key is about to expire.

Goal: Keep DB utilization under 100% through the stampede — without raising DB capacity above 2K req/s.

Use the simulator above with no hints. These checks update live as you play.

10

Interview question

“Explain cache-aside vs write-through, how you'd pick a TTL, and how you'd prevent a cache stampede on a hot key.”

Next: Consistent Hashing