Implementing cache-aside and Measuring the Hit Rate

Goal

Implement cache-aside yourself to confirm the cost difference between a miss and a hit in numbers, and add negative caching and version-key invalidation.

Why it matters

Cache-aside is the most common pattern, so the code can be copied from anywhere. What is actually hard is the details. Whether to update or delete the cache on a write, whether to cache data that does not exist, and what to do when you must invalidate everything at once. First — the reason deletion, not an update, is the right answer is that when two writes interleave, an update can leave an old value, but deletion cannot. Second — traffic that repeatedly looks up keys that do not exist passes through the cache and hits the original directly. This is called cache penetration, and it is blocked with negative caching with a short TTL. Third — when a product category changes wholesale, finding and deleting all related keys is dependency-tracking hell. A version key turns it into bumping a single version number.

Steps

Start /opt/app/slowdb.py on 127.0.0.1:8151. GET /item/1 returns 200 and GET /stats returns {"queries":<n>}.
In /root/ca/cache.py, create get_item(id). On a cache miss, query the original and store it under the item:<id> key with 300 seconds. In /root/ca/miss.out, write source=origin ms=<정수> (ms is an integer), and ms must be at least 100.
Query the same item again. In /root/ca/hit.out, write source=cache ms=<정수> origin_queries_delta=0 (ms is an integer), and ms must be under 50.
/root/ca/update.py updates the original and then deletes the cache key. The source must not contain code that writes the updated value back to the cache. After running it, EXISTS item:1 must be 0.
When you query the nonexistent item:9999, the original returns 404, and you cache that result with a 60-second lifetime. On the second query the origin query count must not increase, and TTL item:9999 must be between 1 and 60 inclusive.
Run 100 lookups (20 unique keys) with /root/ca/bench.py, and in /root/ca/hitratio.txt write hits=<n> misses=<n> ratio=<소수> (ratio is a decimal). The ratio must be at least 0.75.
Change the cache key to the form v<버전>:item:<id> (the version number goes in the first part) and manage the version with the ver:item key. After bumping the version, a lookup must increase the origin query count. In /root/ca/version.out, write old_ver=<n> new_ver=<n> refetched=true.

Notes

Checking the origin query count: curl -s http://127.0.0.1:8151/stats
Set the TTL of negative caching much shorter than a normal cache — because the data may appear soon.
Common mistake 1: updating the cache on a write — an old value can remain under concurrent writes.
Common mistake 2: not doing negative caching, so requests that throw nonexistent IDs hit the original directly (cache penetration).

Start the slow original

Start /opt/app/slowdb.py on 127.0.0.1:8151. GET /item/1 returns 200 and GET /stats returns {"queries":<n>}.

/opt/app/slowdb.py deliberately takes time on every query. It also reports the number of queries it has received.

Build the cache miss path

In /root/ca/cache.py, create get_item(id). On a cache miss, query the original and store it under the item:<id> key with 300 seconds. In /root/ca/miss.out, write source=origin ms=<정수> (ms is an integer), and ms must be at least 100.

If it is not in the cache, make a trip to the original, put the result in the cache, and return it. The first lookup is bound to be slow.

Serve the second lookup from the cache

Query the same item again. In /root/ca/hit.out, write source=cache ms=<정수> origin_queries_delta=0 (ms is an integer), and ms must be under 50.

When you query the same key again, it must not go to the original. Check that the origin query count does not increase.

Delete the cache on update

/root/ca/update.py updates the original and then deletes the cache key. The source must not contain code that writes the updated value back to the cache. After running it, EXISTS item:1 must be 0.

It is deletion, not an update. The reading material explains why. The next lookup fills in the latest on its own.

Negative-cache a missing key

When you query the nonexistent item:9999, the original returns 404, and you cache that result with a 60-second lifetime. On the second query the origin query count must not increase, and TTL item:9999 must be between 1 and 60 inclusive.

If you repeatedly query a key that does not exist, it goes to the original every time. Cache "absent" too, with a short lifetime.

Measure the hit rate

Run 100 lookups (20 unique keys) with /root/ca/bench.py, and in /root/ca/hitratio.txt write hits=<n> misses=<n> ratio=<소수> (ratio is a decimal). The ratio must be at least 0.75.

Count hits and misses separately and compute the ratio. The value varies with the request distribution.

Invalidate everything with a version key

Change the cache key to the form v<버전>:item:<id> (the version number goes in the first part) and manage the version with the ver:item key. After bumping the version, a lookup must increase the origin query count. In /root/ca/version.out, write old_ver=<n> new_ver=<n> refetched=true.

If you embed the version in the key name, bumping only the version means nobody looks up the old keys. You do not have to delete anything yourself.