Caching Strategies Explained: Patterns, TTLs, Pitfalls (2026)

Caching Strategies Explained: Patterns, TTLs, Pitfalls (2026)

Blog Article

Caching Strategies Start With One Question: How Stale Can This Data Be?

Most caching answers jump straight to Redis. The stronger answer starts one step earlier. Caching is a deliberate trade of freshness for speed, so the first thing to establish is how much staleness the product can tolerate. A product description can be five minutes old. An account balance after a transfer usually cannot be stale at all.

Once staleness has a number, the rest of the design follows from it: which pattern to use, how long entries live, how they are invalidated, and what happens when the cache is empty or wrong. This guide walks through those decisions in the order an interviewer expects to hear them.

Where Caches Actually Sit in a Request Path

A single page load usually passes through four or five caches before anything touches a database. Naming them early shows you are designing a path, not adding a box.

• Browser cache: the cheapest hit available, controlled by Cache-Control and ETag headers. Nothing leaves the device.
• CDN or edge cache: serves static assets and cacheable API responses close to the user, which removes both origin load and network distance. This is covered in more depth in the guide to how a CDN works in system design.
• In-process cache (L1): a small map inside the application instance. Nanosecond reads, but every instance holds its own copy, so consistency across instances is loose.
• Distributed cache (L2): Redis or Memcached, shared by all instances. Sub-millisecond reads over the network, one shared view of the data.
• Database buffer pool: the database already caches hot pages in memory. A cache in front of it is only worth adding when the query itself, not the disk read, is the expensive part.

That last point matters in interviews. A cache is not a repair for a missing index or an N+1 query pattern. Fix the query first, using database indexing, and cache what remains expensive.

The Four Caching Patterns and When Each Fits

Every caching pattern answers the same two questions differently: who populates the cache, and when does the database learn about a write?

Cache-Aside (Lazy Loading)

Cache-aside is the default for read-heavy workloads. The application owns the logic and the cache holds only what has actually been requested.

1. Read the key from the cache.
2. On a hit, return the value and stop.
3. On a miss, query the database.
4. Write the result into the cache with a TTL.
5. Return the value to the caller.
6. On an update, write to the database and then delete the key rather than overwriting it.

Deleting on write is safer than updating on write. Two concurrent updates can interleave and leave a stale value permanently cached, while a delete forces the next reader to load the current row.

Write-Through

Write-through writes to the cache and the database in the same operation, so a successful write leaves both in agreement. Reads are consistently fast and the code is simple. The cost is write latency, plus a cache full of data nobody has asked for. It suits small, hot datasets such as user profiles or feature flags.

Write-Behind (Write-Back)

Write-behind acknowledges the write as soon as the cache accepts it and flushes to the database asynchronously, usually in batches. It absorbs write bursts extremely well, which is why it is used for counters, view tallies, and telemetry. It also means an acknowledged write can be lost if the cache node dies before the flush. Never use it for money, orders, or anything a user is told has been saved.

Refresh-Ahead

Refresh-ahead renews an entry in the background before it expires, so hot keys never present a miss to a user. It works well for a small, predictable set more info of popular keys, such as the homepage feed or a top-100 leaderboard. Applied to a large key space it wastes work refreshing entries nobody reads again.

How Do You Choose a TTL?

A TTL is not a number you pick by feel. It comes from three inputs: how stale the data may be, how often it changes, and how expensive a miss is.

Work an example. A product-detail endpoint serves 50,000 reads per second. At a 95% hit ratio, the database sees 2,500 reads per second. Let the hit ratio slip to 80% and the database now sees 10,000 reads per second, four times the load, from the same user traffic. That sensitivity is the reason hit ratio belongs on a dashboard next to latency and throughput, not in a quarterly report.

Two rules keep TTLs honest. Give every entry a small random jitter, for example 300 seconds plus or minus 30, so a batch of keys written together does not expire together. And set the TTL below the point where a stale value would be visibly wrong, not merely below the point where it feels acceptable.

Invalidation: The Part Most Designs Skip

A cache without a stated ownership rule slowly becomes a second, inconsistent database. Decide before launch which of these applies to each key family.

• TTL only: simplest, and correct when bounded staleness is genuinely acceptable. Say the bound out loud: "prices may be up to 60 seconds old."
• Event-driven invalidation: the write path publishes a change event and the consumer deletes the affected keys. Accurate, but it needs a reliable event path such as a transactional outbox, or invalidations get lost exactly when data changes.
• Versioned keys: embed a version or updated-at value in the key, for example product:1234:v9. Nothing is ever invalidated, because a write simply produces a new key and the old one ages out. This sidesteps most race conditions at the cost of extra memory.

Whichever you choose, write down which service owns each key. Shared keys with no owner are how two teams end up caching the same entity under different rules.

Want to master this with video lessons and real case studies? This topic is covered in depth in my Udemy course System Design Fundamentals for Interviews — 5.5 hours, rated 4.8★, built from real interview questions.

Sizing the Cache and Picking an Eviction Policy

Memory is finite, so eviction is not an edge case. Size the cache from the working set, meaning the keys actually read within one TTL window, not from total data volume. If 500,000 products are hot and each cached record is roughly 2 KB, the working set is about 1 GB, and a 2 GB instance leaves room for growth and overhead.

• LRU (least recently used): the sensible default. It matches workloads where recent access predicts the next access.
• LFU (least frequently used): better when a stable set of items stays popular for weeks and you do not want a nightly batch job to flush them.
• FIFO: cheap and predictable, but it evicts hot entries purely because they are old. Use it only for uniform, short-lived data.
• TTL-based expiry: not really an eviction policy. It bounds staleness; it does not bound memory. You need both.

Watch the eviction rate as closely as the hit ratio. A high hit ratio with heavy eviction means the cache is thrashing and one traffic shift away from a much worse number.

The Four Ways Caches Fail in Production

Interviewers reward candidates who can name these before being asked.

1. Cache stampede: a popular key expires and every concurrent request misses at once. A key serving 5,000 requests per second becomes 5,000 simultaneous database queries. Fix it with request coalescing, so one caller loads the value while the rest wait, plus TTL jitter and background refresh for the hottest keys.
2. Cache penetration: requests for keys that do not exist never populate the cache, so every one reaches the database. Fix it by caching the negative result with a short TTL, and by putting a Bloom filter in front when the key space is attacker-controlled.
3. Hot keys: one celebrity record concentrates traffic on a single cache shard while the rest idle. Fix it with a short-lived in-process L1 cache, key splitting across several suffixes, or read replicas of that shard.
4. Cold start after a restart: an empty cache sends 100% of traffic to a database sized for 5%. This is how a cache outage becomes a full outage. Fix it with staged warm-up, rate limiting on the origin path, and load tests run with the cache deliberately disabled.

The last one is worth stating explicitly in any design: a cache that the system cannot survive without is a dependency, not an optimization, and it needs the same availability planning as the database behind it.

How to Talk About Caching in a System Design Interview

Keep a 30-second answer ready:

"This is a read-heavy path with roughly a 100:1 read-to-write ratio, so I would put a cache-aside layer in Redis in front of the database. Keys are per-entity, TTL around five minutes with jitter, and writes delete the key rather than update it. I would add request coalescing to prevent stampedes and negative caching for missing IDs. I would track hit ratio, eviction rate, and origin load, and I would load test with the cache off so I know what happens when it restarts."

Then expand in this order if the interviewer pushes: staleness budget, pattern choice, TTL and invalidation, memory sizing and eviction, failure modes, metrics. Naming the trade-off is what scores. Redis and Memcached are implementations, not answers.

Key Takeaways

• Start from a staleness budget. Every other caching decision follows from it.
• Cache-aside plus delete-on-write is the safe default for read-heavy systems.
• Write-behind is fast and lossy. Keep it away from anything a user is told was saved.
• Size the cache from the working set, jitter your TTLs, and watch eviction rate alongside hit ratio.
• Plan for a cold cache. An unavailable cache should degrade the system, not take it down.

Next Steps

A cache only postpones pressure on the layer underneath it. Continue with database replication, sharding, and consistency to design the durable store that absorbs every miss, and see how replica lag and cache staleness combine into the consistency model your users actually experience.

Report this page