Caching Layers
In a high-traffic full-stack environment, relying solely on the primary database for every single read request is a recipe for system collapse. Caching is the architectural practice of storing copies of frequently accessed data in a temporary, high-speed storage layer. By serving data from memory rather than executing complex disk reads, you drastically reduce network latency and protect your primary relational database from overwhelming load. Proper caching stabilizes your application infrastructure by buffering load.
The caching spectrum
Data can be cached at multiple points along the request lifecycle. A professional architecture utilizes a multi-tiered approach, ensuring the request travels the shortest possible distance before finding the data it needs.
- Client-side caching: The user’s browser stores static assets and API responses locally. This is the fastest possible cache because the request never leaves the device. It is controlled via the HTTP
Cache-Controlheaders sent by your server. - Edge caching: A Content Delivery Network (CDN) stores static assets and sometimes rendered HTML on servers physically located near the user. If a user in Tokyo requests an image hosted on a server in New York, the CDN serves a cached copy from a data center in Japan, cutting latency by hundreds of milliseconds.
- Server-side caching: The backend utilizes an in-memory data structure store, such as Redis or Memcached, to hold the results of expensive database queries or third-party API calls. This prevents the database from performing the same complex calculation twice.
Cache invalidation strategies
Phil Karlton famously noted that cache invalidation is one of the hardest problems in computer science. If the underlying data in your database changes but the cache is not updated, your application serves stale, incorrect information to the user.
- Time-To-Live (TTL): The cache is configured to expire automatically after a set duration, such as five minutes. This is ideal for data that changes frequently but does not require absolute real-time accuracy, such as a social media feed or a public product catalog.
- Write-through caching: Every time the application writes new data to the database, it simultaneously updates the cache. This ensures the cache is always perfectly synchronized, but it adds network latency to your write operations.
- Cache-aside: The application checks the cache first. If the data is missing, it fetches the data from the database, stores a copy in the cache for future requests, and then returns the response.
Learning Point: Never cache highly sensitive, user-specific data like passwords, session tokens, or credit card numbers in a shared server-side cache without strict access controls. If a key collision occurs due to a programming error, you risk exposing one user’s private data to another user.
Implementing the cache-aside pattern
The cache-aside pattern is the most common strategy for full-stack API development. It safely isolates the cache failure. If the Redis server crashes or runs out of memory, the application simply falls back to querying the primary database directly, ensuring the system remains available, even if it runs a little slower.
/**
* A service function demonstrating the cache-aside pattern.
* It attempts to read from Redis first, falling back to PostgreSQL
* if the cache is empty, and then populates the cache for the next user.
*/
const getUserProfile = async (userId) => {
const cacheKey = `user:profile:${userId}`;
// 1. Attempt to fetch the data from the high-speed cache
const cachedData = await redisClient.get(cacheKey);
if (cachedData) {
return JSON.parse(cachedData);
}
// 2. Cache miss occurred, fetch from the primary database
const databaseRecord = await db.users.findById(userId);
if (!databaseRecord) {
throw new Error("User not found.");
}
// 3. Store the result in the cache with a 1-hour Time-To-Live (TTL)
await redisClient.set(
cacheKey,
JSON.stringify(databaseRecord),
{ EX: 3600 }
);
return databaseRecord;
};
Eviction policies and memory management
Unlike traditional hard drives, in-memory caches have strictly limited storage capacities. When the cache reaches its maximum memory limit, it must decide which data to delete to make room for new entries. This decision is governed by an eviction policy.
- Least Recently Used (LRU): The system discards the items that have not been accessed for the longest period of time. This is the most common and generally effective policy for standard web applications.
- Least Frequently Used (LFU): The system tracks how many times an item is accessed overall and discards the items with the lowest hit counts. This is highly useful for static catalogs where certain items are definitively more popular than others over long periods.
- First In, First Out (FIFO): The system simply deletes the oldest entries first, regardless of how often they are accessed. This is rarely used in performance-critical applications because it might evict a highly popular piece of data simply because it was cached a long time ago.
Mitigating the thundering herd problem
A severe architectural failure occurs when a highly popular cache key expires during a period of peak traffic. If ten thousand users request the application homepage at the exact second the cached version expires, all ten thousand requests will bypass the cache and hit the database simultaneously. This phenomenon is known as the thundering herd problem, and it can easily crash a robust database.
- Mutex locks: When a cache miss occurs, the first request acquires a system lock. The other concurrent requests wait for a few milliseconds until the first request populates the cache, at which point they all read the newly cached data safely.
- Stale-while-revalidate: The server serves the expired cached data to the user immediately, while a background worker process quietly fetches the fresh data from the database and updates the cache for the next user.
- Jitter: Adding a small, randomized amount of time to your TTL values. If all your cache keys for a specific data type expire at exactly midnight, your database will experience a massive spike in traffic at 12:01 AM. Jitter spreads those expirations out over a wider, randomized window of time.
Note: Caching is often used to mask poor database performance. Before implementing a complex Redis layer, ensure you have properly optimized your SQL queries and added the correct database indexes. Caching should enhance a fast system, not rescue a broken one.
By strategically layering these caching mechanisms and understanding how data expires, you protect your infrastructure from unpredictable traffic spikes. This guarantees that your frontend engine receives data with the lowest possible latency, maintaining a seamless experience for the end user.
Popular Guides