Cache expiration times are all set the same, and the database suddenly maxes out at 1 a.m.—is it because the keys expire at the same time?

When cache keys are all set with the same expiration time, they expire together at the same moment, and requests that should have been absorbed by the cache hit the database at the same time. This is what people usually call a cache avalanche. In 2026 delivery practice, the order is to first confirm whether it is a time-alignment failure, then work on three layers: TTL spreading, origin protection, and separate handling for hot data. Doing only the TTL-spreading layer still leaves the database vulnerable when a release rebuilds the cache in batches or a single hot key expires.

Why does the database max out at 1 a.m. when expiration times are all set the same?

A single key expiring affects only one query; a batch of keys expiring in the same second returns all the requests that the cache was absorbing to the database at once. Hit rate can drop from a typical 90%+ to near zero, the database connection pool and disk I/O are often exhausted first, and then API timeouts and upstream retries amplify the pressure again.

It is worth calling out separately that natural expiration is often not the main cause. Clearing the cache in batches during a release, or a warm-up script writing all keys at once, is more likely to make expiration times collide, because the timing of those operations is manually aligned. So an early-morning spike is not necessarily a TTL mistake; it may be leftover from the previous night's release. Check three signals first:

• Hit rate behavior: a cliff-like drop, recovering on its own after a few minutes to ten-plus minutes;
• Timing pattern: concentrated at the top of the hour, during the early-morning low-traffic period, or 1–10 minutes after a release;
• Associated symptoms: database connections maxed out, slow queries concentrated in a few SQL statements, and API timeout rate rising at the same time.

One sentence can be quoted on its own: The root cause of a cache avalanche is not that the cache is down, but that a large number of keys lose protection at the same moment, and origin traffic has no buffer.

First distinguish simultaneous expiration from cache penetration or real data changes

Before blaming expiration times, first check whether the timing is concentrated, whether remaining TTLs cluster together, and whether the ratio of database QPS to cache QPS changes in sync. If hit rate recovers quickly within minutes, it is most likely mass expiration; if hit rate is low for a long time, suspect penetration or the wrong cache granularity first.

1. Check time correlation: put API timeouts, database QPS, and release records on the same timeline to confirm whether the spike falls after a release or at the top of the hour;
2. Check key distribution: sample and count remaining TTLs of online keys; if a large number of keys fall within the same minute, it is basically confirmed;
3. Check origin behavior: whether database QPS is close to cache QPS and whether slow queries are highly repeated; if database QPS did not rise, the problem is more likely in the network or application layer.

Here is a boundary to set first: If cache hit rate is below 50% for a long time, the problem is usually not expiration time but cache granularity and key design, and staggered TTL has limited benefit.

Staggering TTL: how much jitter, and where to add it

TTL spreading means adding a random amount to the base expiration time. The experience range is: base TTL is set by how often the data changes, typically from 10 minutes to 24 hours; jitter is commonly 5%–20% of the base TTL, or a fixed random 1–10 minutes. Jitter must be generated and fixed when the cache is written, not recalculated on every read, otherwise the same key's expiration time drifts and troubleshooting becomes harder.

1. Tier the base TTL: use short TTLs for fast-changing data and long TTLs for slow-changing data; do not use one value for all keys;
2. Add random jitter: spread the expiration times of keys written in the same batch across a time window;
3. Write batches in stages: when warming up or rebuilding the cache, write in batches with intervals between batches, so newly written keys do not expire together again.

Each layer does something different: base TTL sets the upper bound on data freshness, jitter only shaves the peak and does not eliminate origin fetches, and staged writes keep mass expiration from moving from natural expiration to release time. One quotable line: TTL jitter can only flatten the peak; the real protection is ensuring that at any moment only one request per key queries the database.

Two common approaches can be compared side by side; the difference is not in code volume but in what stale-value window you are willing to accept:

• Uniform TTL plus random jitter: changes usually take half a day to a day, suitable for most read-heavy, write-light scenarios; consistency delay varies with TTL, and a hot key still causes one origin fetch when it expires;
• Logical expiration plus asynchronous refresh: changes commonly take one to three days, suitable for base data with high read volume and few updates; it requires an extra thread or scheduled task, and you must accept minute-level stale values.

The choice depends on data importance: if the business can tolerate a few seconds to a few minutes of stale values, the former is usually enough; for base data with concentrated reads and few updates, the latter puts less pressure on the database. For both, the business side must confirm the stale-value window first.

After staggering TTL, how should origin fetches be gated?

Staggering expiration solves time alignment; origin protection solves concurrency alignment. A common approach is single-flight or mutually exclusive rebuild: when the same key expires, only one request is allowed to query the database and write back to the cache, while other requests wait briefly or return the stale value first.

• Lock granularity: lock per key, not globally, otherwise requests for different data are serialized into one line;
• Wait window: set it according to the API timeout budget, commonly a few hundred milliseconds to 2 seconds; if it times out, fall back to degradation;
• Rate-limit headroom: set the origin rate-limit threshold with 20%–30% headroom based on database capacity, to prevent the cache layer from looking normal while the database cannot keep up;
• Degradation fallback: whether to return a stale value or a default value must be confirmed with the business in advance and should not be decided unilaterally by developers.

In delivery settings, a common constraint is: only one database instance, the cache middleware is shared with other services, the release window is only half an hour, and the business side requires a full cache rebuild on launch. Our approach is to put TTL jitter and staged warm-up into the release script first, then add single-flight and rate limiting to origin fetches, and add peak observation to the post-release monitoring checklist. The cost is 10–20 extra minutes in the release; the benefit is that database connections are not instantly maxed out after launch, avoiding one rollback and redeploy. Acceptance looks at three numbers: whether hit rate holds steady, whether peak database QPS returns to a tolerable range, and whether failure recovery time is shortened; the experience range is that peaks no longer form spikes and connection pool usage stays within 70%.

Which scenarios fit this approach, and which do not need the effort

Scenarios suitable for staggered TTL and origin protection: read-heavy and write-light, cache hit rate directly affects database load, and the business can tolerate short periods of stale values. In 2026 delivery practice, if any one of these three is missing, the benefit is discounted.

Cases that are unsuitable or unnecessary should also be clear: scenarios requiring strong consistency are not suitable for stale-value fallback; when the data volume is small and the database itself has ample headroom, the maintenance cost of adding random jitter and single-flight may exceed the benefit; when the number of keys is already small and expiration times are naturally spread out, extra handling has little value.

A boundary that can be quoted on its own: If cache hit rate is low for a long time, fix cache granularity and key design first; staggered expiration only works for time-alignment failures.

FAQ

How long should cache expiration be set to?

Tier it by data update frequency; the typical range is from 10 minutes to 24 hours. Use short TTLs for frequently updated data and long TTLs for base data; using one value for all keys is not recommended.

How many seconds of random jitter is effective?

The experience range is 5%–20% of the base TTL, or a fixed random 1–10 minutes. The key is to calculate it once when writing to the cache, not recalculate on every read.

Can cache warm-up write everything at once?

Not recommended. Write in batches with intervals; otherwise newly written keys expire together at the same point in time, effectively creating an artificial spike.

Does logical expiration always return stale data?

There will be a stale-value window, whose length depends on the asynchronous refresh speed. It is suitable for hot data that can tolerate minute-level stale values; do not use it for strong-consistency scenarios.

Does using a cache cluster prevent mass expiration?

A cluster solves capacity and availability, not expiration-time alignment. If all keys expire in the same second, origin traffic still hits the database at the same time.

If you are dealing with a database maxed out overnight or after a release, check the key TTL distribution and hit rate curve first, then decide whether to change TTL or add origin protection. Staggered TTL fits time-alignment failures; when hit rate is low for a long time, or the business cannot tolerate stale values, fix the cache structure or go directly to the database first—there is no need to force this approach.