Working With TTL and Expiry Policy

Goal

Understand exactly how TTL behaves, reproduce and block the trap where a TTL silently disappears, and confirm the differences between eviction policies through actual eviction.

Why it matters

Most of the "Redis memory keeps growing and I don't know why" inquiries are keys without a TTL. And the reason it is not noticeable is that even though the code uses SETEX properly, somewhere a SET updates only the value and wipes out the lifetime that was there. Step 3 of this lab is exactly that scene. Another thing to take away is the fact that expiration is not immediate. Redis deletes when a key is accessed, or samples keys in the background and deletes them. So a key whose expiration time has passed can remain in DBSIZE, and that is the answer to "it expired, so why hasn't memory gone down?". The audit script at the end is something you can take to production as is.

Steps

Put cache:a in with a value and a 60-second lifetime, and in /root/ttl/ttl1.txt write ttl=<남은초> (the remaining seconds). The value is between 1 and 60 inclusive.
Remove the TTL of cache:a. /root/ttl/ttl2.txt must contain ttl=-1.
Put cache:b in with a 60-second lifetime, then update only the value with SET cache:b <새값> (the new value). Record in /root/ttl/lost.txt that the TTL becomes -1, and write the result of trying again with KEEPTTL to /root/ttl/kept.txt as ttl=<양수> (a positive number).
Put cache:c in with a 1-second lifetime and wait 2 seconds, then write the DBSIZE before it has been accessed and the DBSIZE after access to /root/ttl/lazy.txt as before=<n> after=<n>.
Set maxmemory-policy to allkeys-lfu. /root/ttl/policy.txt must contain maxmemory-policy allkeys-lfu.
Lower maxmemory to 8mb and put in enough keys with /root/ttl/fill.py to cause eviction. In /root/ttl/evicted.txt, write evicted_keys=<n>, and n must be greater than 0.
With /root/ttl/audit.sh, iterate with SCAN and write the keys whose TTL is -1 to /root/ttl/nottl.txt. The script must not contain the string KEYS, and the result file must have at least 1 key.

Notes

Checking the lifetime: TTL key — -1 means no expiration and -2 means no such key.
Updating while keeping the TTL: SET key val KEEPTTL
Eviction statistics: redis-cli INFO stats | grep evicted_keys
TTL jitter: ttl = base + random(-spread, +spread) spreads out simultaneous expirations.
Common mistake: omitting the TTL on a cache key under the volatile-lru policy — that key is never evicted.

Set a TTL and check the remaining lifetime

Put cache:a in with a value and a 60-second lifetime, and in /root/ttl/ttl1.txt write ttl=<남은초> (the remaining seconds). The value is between 1 and 60 inclusive.

There is a command that sets the lifetime as you insert, and another that sets it on an existing key. Look up the remaining lifetime.

Remove the TTL to make it a permanent key

Remove the TTL of cache:a. /root/ttl/ttl2.txt must contain ttl=-1.

The lookup result for a key with no TTL is -1. Be careful: -2 means something else.

Reproduce and block the trap where SET wipes out the TTL

Put cache:b in with a 60-second lifetime, then update only the value with SET cache:b <새값> (the new value). Record in /root/ttl/lost.txt that the TTL becomes -1, and write the result of trying again with KEEPTTL to /root/ttl/kept.txt as ttl=<양수> (a positive number).

You only updated the value, yet the lifetime disappears. There is an option that preserves the lifetime.

Observe that expiration is not an immediate return of memory

Put cache:c in with a 1-second lifetime and wait 2 seconds, then write the DBSIZE before it has been accessed and the DBSIZE after access to /root/ttl/lazy.txt as before=<n> after=<n>.

A key whose expiration time has passed but which nobody has accessed remains for a while. Compare the counts before and after the lookup.

Set the eviction policy

Set maxmemory-policy to allkeys-lfu. /root/ttl/policy.txt must contain maxmemory-policy allkeys-lfu.

The default does not suit cache use. See the reading material for which policy is good for a dedicated cache.

Make eviction actually happen

Lower maxmemory to 8mb and put in enough keys with /root/ttl/fill.py to cause eviction. In /root/ttl/evicted.txt, write evicted_keys=<n>, and n must be greater than 0.

Set a small memory limit and put in many keys. The eviction statistics are in INFO.

Build an audit script for keys without a TTL

With /root/ttl/audit.sh, iterate with SCAN and write the keys whose TTL is -1 to /root/ttl/nottl.txt. The script must not contain the string KEYS, and the result file must have at least 1 key.

Iterate with SCAN and collect the keys whose remaining lifetime is -1. It must be in a form that you can run periodically in production.