LRU Caching Practice

In this article, let’s practice applying LRU caching in an enterprise application for different business cases.

Background

Firstly, let’s assume we have a User table in database and there may be millions of rows in this table. LRU caching is only meaningful if the total data to be cached is huge, which means it is impossible to put all the data in memory at same time. For example, if the rows in User table are only thousands, we could consider cache everything, but if it is millions, it becomes unsuitable to cache everything, right?

Our target is to LRU cache the total huge user data in our enterprise application for different business cases.

Practice

Case 1 Users are accessed by ID only, it is a single server application(APP1) and the only way to change the user data is through this application.

In this case, since “the only way to change the user data is through this application”, items cached could always be kept active in the cache, but each time a user is modified, and the item is in the cache, it should be set expired.

So the most suitable implementation for this case is to use in an in-process LRU hash table in APP1 for caching users. The LruDictionary class is a thread safe LRU hash table implementation provided by NIntegrate.

Case 2 Based on Case 1, additionally, user data may be read by another applications(APP2) deployed on a different server, but the access from APP2 is not very often.

In this case, since “the access from APP2 is not very often”, we could consider simply expose the LruDictionary instance for caching users in Case 1 as a WCF service which is still be deployed in APP1. This service should contain necessary operation methods for get/set the user cache.

Case 3 Based on Case 1, additionally, user data may be read by another applications(APP2) deployed on a different farm consisted of 3 servers, and the access from APP2 is much more often than from APP1 itself.

In this case, since “the access from APP2 is much more often”, still keep the LRU hash table implementation in APP1 becomes not reasonable. Because the load pressure from APP2 may kill the APP1 server.

So we should consider introduce some distributed hash table implementation such as memcached.

Case 4 Users are accessed by ID only, it is a single server application(APP1), but user data in the table could be modified by out-of-our-control applications.

Case 1-3 all could ensure “the only way to change the user data is through APP1”, that is a required condition for keeping cached users always be updated and consistent with data in database. But in this case, since “user data in the table could be modified by out-of-our-control applications”, it becomes impossible to keep “cached users always be updated and consistent with data in database”.

So if your business still require this, you have to choose not to do any caching for user data. Or, if data inconsistent for short time could be tolerated, you could consider using a LRU hash table supporting “item expire time”, which could automatically expire items when the expire time elapsed. The LruItemCache class is a thread safe LRU hash table implementation provided by NIntegrate which supports “item expire time” and more other features.

Case 5 Based on Case 1, additionally, users depend on its parent department. If a department is modified, all its child users in cache should be removed.

In this case, compared to case 1, we need a LRU hash table supporting “dependency based item expiration”. The LruDependingDictionary class is an extended version of LruDictionary provided in NIntegrate, which supports “dependency based item expiration”, the LruItemCache class provided by NIntegrate could also has this ability.

Case 6 Same to Case 5 except that users are accessed not only by ID, but also by unique user name, or user email address.

In this case, compared to case 5, because the hash table hashes items by the key value, for one hash table, if we choose ID as key, user name, user email could not be the key, which means the cache using ID as key could not benefit user name or user email based user data access.

To support this case, if you use 3 hash tables, each one use ID, user name or user email as key, the problem is there may be duplicated data cached in memory. A better solution is use a LRU hash table supporting indexing, which means for each user, there is still only one copy cached in memory, but cached items could be accessed through different indexes (indexed by different keys). The LruItemCache class provided by NIntegrate just implements this, it is not only a thread safe LRU hash table, but also supports “item expire time”, “dependency based item expiration” and multiple indexes for cached items.