Caching

Caching means keeping a copy of data so it can be found faster later. A result can be saved after a database query. When the same result is needed again, it can be read from the cache instead of the database.

For example, a list of popular posts may be requested many times. The list may change only a few times a day. A cached copy can answer most requests, while the database is used when a fresh copy is needed.

Why is caching used?

• Faster responses: A stored result can be returned without repeating slow work.
• Less work for the database: The same query does not have to run for every request.
• Better handling of busy periods: Many requests can use one stored result.

Caching also creates a problem: the copy can become old. A good cache plan must say when data is stored, when it is changed, and when it is removed.

Cache hits and misses

A cache hit means that the value was found in the cache. The stored value is returned.

A cache miss means that the value was not found. The value is fetched from its main source, such as a database. It can then be saved in the cache for the next request.

Common caching patterns

Cache-aside

With cache-aside, the application manages the cache:

1. The cache is checked first.
2. If the value is found, it is returned.
3. If the value is missing, it is read from the database.
4. The value is saved in the cache and returned.

Only data that is requested is added to the cache. The first request is still slow because it must read from the database.

Read-through

With read-through, the application reads through a cache service. When a value is missing, that service reads it from the database, saves it, and returns it. The steps are close to cache-aside, but the cache service handles the miss.

Write-through

With write-through, the database is updated first. The new value is then added to the cache. The next read can find it there.

Each write takes more work. If one write succeeds and the other fails, the two copies may be different. Data that is written but never read can also take up cache space.

Write-around

With write-around, new data goes to the database but is not added to the cache at once. It is cached only if it is read later. This saves cache space for data that may never be read. The first read after the write will be a cache miss.

Any old cached copy must still be removed. Otherwise, the old value could be returned.

Write-behind

With write-behind, new data is written to the cache first. The database is updated later in the background. Writes can be faster, but data can be lost if the cache fails before the database is updated. This pattern needs a plan for failures and recovery.

These patterns can be used together. Cache-aside can fill the cache when a value is read. Write-through can update it when the database changes. Another choice is to delete the old cache entry after a database write and let the next read fill it again.

Expiration, invalidation, and eviction

These three words describe different reasons for removing data from a cache:

• Expiration: A value is removed because its allowed time has ended.
• Invalidation: A value is removed because the original data has changed.
• Eviction: A value is removed because the cache needs space.

The allowed time for a cached value is called TTL, or time to live. A short TTL gives fresher data but causes more database reads. A long TTL reduces database work but can return old data for longer.

Cache invalidation

If a post is edited, its cached copy may no longer be correct. Several ways can be used to deal with this.

Delete after a write: First, the database is updated. Then the cache key is deleted. The next read gets the new value from the database and saves it in the cache. This is often simple to use with cache-aside. If the delete fails, the old value may remain until its TTL ends.

Update after a write: The database and cached value are both updated. The next read can be a cache hit with new data. This becomes harder when the same data appears in many cached pages or lists.

Wait for the TTL: No immediate change is made to the cache. Old data may be shown until the TTL ends. This is useful only when a short delay is acceptable.

Use a version in the key: A key such as post:42:v3 can be changed to post:42:v4 after an update. New reads use the new key. Old keys still need to expire or be removed later.

Return old data during a refresh: An old value can be returned for a short time while a new one is loaded. This can keep a busy page fast, but it should be used only when old data is acceptable.

Invalidation is easy when one value has one key. It is harder when one change affects many cached results. A new post may change a recent-posts list, a tag page, and an account page. Each affected result must be found.

Cache eviction

A cache has limited memory. When it becomes full, an eviction policy decides which value should be removed.

• LRU (least recently used): Removes values that have not been read recently.
• LFU (least frequently used): Removes values that have been read the fewest times.
• FIFO (first in, first out): Removes the value that was added first.
• Random: Removes a value without looking at how often it is used.
• No eviction: Keeps existing values and rejects new cache writes when memory is full.

LRU can work well when recent reads are a good sign of future reads. LFU can help when some values stay popular for a long time. The best choice depends on how the cache is used.

Some caches can evict any key. Others can evict only keys that have a TTL. In the second case, memory can fill up if too few keys are allowed to be removed.

Cache keys

A cache key is the name used to find a stored value. It must include every part of a request that changes the answer.

For one public post, post:42 may be enough. For a list, the page number, filters, and sort order may also be needed. For private data, the account ID must be included. Otherwise, one account could receive data cached for another account.

Two requests should use the same key only when the same answer can safely be returned to both.

What should be cached?

Data that is read often but changes slowly is a good choice. A slow database query or a result that takes time to calculate may also be worth caching.

Data that changes on every request may not benefit much. Data that must always be current should be cached only when it can be updated or removed at the right time.

The main data source must remain clear. If a read cache is empty or unavailable, the value should still be available from its main source.

Where can a cache live?

An in-process cache is kept inside one running application. It is fast, but each application instance has its own copy. A change in one instance does not update the others by itself.

A shared cache, such as Redis, can be used by many application instances. The same keys are available to all of them. It also needs a network call, and the cache service can fail.

Browsers and CDNs can cache web responses close to readers. Each cache layer needs its own rules for old data and updates.