Three Write Strategies, and Stampedes
Summary
Read strategies and write strategies are different axes. And the paradox that the more popular a key is, the more dangerous its expiration moment is, is the heart of the stampede.
Why this was needed
There are three names people mix up when talking about cache patterns: cache-aside, write-through, and write-behind. These are not competing choices but answers to different questions. The first answers who fills on reads, and the other two answer how writes are propagated.
In cache-aside, the application manages the cache directly. When reading, it looks at the cache, and if the value is absent, it reads from the original and puts it in the cache. When writing, it updates the original and deletes the cache entry. It is important that it is deletion, not an update — if two writes interleave, an old value can remain in the cache, but deletion does not carry that risk.
In write-through, a write updates the cache and the original at the same time. The cache is always current, so reads are always fast. In exchange, writes get slower, and data that will not be read for a while is also filled into the cache, wasting space.
In write-behind, you write only to the cache and apply it to the original later in a batch. Writes are very fast and you can gather several writes, but if the cache dies before it is applied, the data is lost. Use it only when you can tolerate loss or have a separate durability mechanism.
How it works
Here are the three strategies in one table.
| Strategy | Write speed | Read freshness | Risk of loss | Typical situation |
|---|---|---|---|---|
| cache-aside | Average | Fresh after a miss | Low | General purpose, read-heavy |
| write-through | Slow | Always fresh | Low | Freshness matters |
| write-behind | Very fast | Always fresh (by the cache) | High | Write bursts, loss tolerated |
And the most notorious trap in caching is the stampede. At the instant a popular key that was being read thousands of times per second expires, thousands of requests experience a miss at the same time, and all rush to the original to recompute the same value. A lookup that one would have sufficed for explodes into thousands, and the original is crushed. In the worst case, the original dies, which causes more misses, and everything collapses in a chain.
The paradox is this — the more popular an item, the more dangerous it is. This is because the more often a value is read, the larger the simultaneous misses at its expiration moment.
There are four mitigations. First, locking or single-flight — on a miss, only the first request queries the original and the rest wait. Second, TTL jitter — it spreads out expiration times. Third, stale-while-revalidate — even after expiration, you return the old value immediately and refresh only once in the background. Fourth, probabilistic early expiration — the closer expiration gets, the higher the probability of refreshing in advance.
In practice you combine them. You spread expirations with jitter, remove waiting with stale-while-revalidate, and gather the refresh into one with a lock.
What you meet in the field
When you use a lock, there are two things you must take care of together. One is putting a TTL on the lock so that it is released even if the owner dies, and the other is having an ownership token so that you do not release someone else's lock by mistake. If you omit the latter, then after A's lock expires by TTL and B takes it, A wakes up late and releases B's lock.
How to design invalidation
The hardest problem in caching is not filling but deciding when to discard. There are three methods, and each bears a different complexity.
Discard by time (TTL). It is the simplest and sufficient in most cases. If you decide "how stale a value are we allowed to show", that is the TTL. However, this question is a business decision, not a technical one, so if a developer decides it alone and moves on, it later becomes "why can't I see what I just changed?".
Delete when it changes. The code that modifies the original also deletes the related cache keys. Freshness improves, but the problem is that you have to know all those keys. When one product changes, the product detail, the list, the search results, and the recommendation list all go stale, and it is hard to keep this relationship, scattered throughout the code, without missing anything.
Invalidate everything at once by version. You put a version number in the key and bump it when something changes. Nobody looks up the old keys any more, so they disappear naturally through the TTL. It is much sturdier because you do not have to track individual keys, but too much becomes invalid at once, so misses crowd in right after. The stampede we saw earlier happens exactly then.
In practice, you usually lay a TTL as the base and add deletion only to the things that really must be reflected immediately. If you try to attach deletion to everything, it cannot be maintained, and if you leave everything to TTL alone, important changes show up late. Which data belongs on which side is something to decide together with the business owner, and if you write that decision down in a table, half of the later cache-related incidents disappear.
Finally, the service must keep running without the cache. If the original cannot withstand the load when you empty the cache entirely, then the cache has become part of the original, not a cache. In this state, a cache failure is a service failure, so you should at least know how much the original can bear.
What you will do in the next lab
You implement cache-aside against a slow original, measure the hit rate, and invalidate everything with a version key. Then, in a separate lab, you actually cause a stampede and attach the four defenses one by one to compare the number of origin calls in numbers.