Cache – Implementation

August 27, 2006

In my previous post on Cache concepts I had indicated that cache are defined by how we want to expire them. I also talked about granularity of cache.

In this post – I will look at where to keep the cache .

When it comes to caching implementation – the consideration are : Where should we keep the cache, how do we access the cache and how do we keep the cache updated. Accessing and updating cache will form topics for subsequent posts.

Where to keep the cache.

We all agree that the closer the cache is to the consumer, the more optimal it is.
Hence – for a browser based application- the most optimal way would be to cache it in user browsers. This can be done by having simple Get URLs and by specifying appropriate meta tags and HTTP headers indicating a cache expiry time.
The problem with this is that it becomes unpredictable – whether the browsers cache or not and whether they actually expire cache when we want them to depends on the browsers and the user settings. This is however by all means is favorable for items which are not subject to change ( typically images do not change even if text of an article does). The second problem with this is that this is cached per user, and not across users.

The second closest place is proxy servers. The proxy servers have the same advantages and problems as the browsers and I will not go into it. Typically, we want to defeat the browser and proxy cache – and I would probably write a separate post on that.

The next nearest location is the Web servers ( I will go ahead and make an assumption that we are talking about a 3 tier application which is clustered at each level.).
Now here we have an option that we use a daemon job to act as a user, access the dynamic page and place the resiting page as a static HTML page. The links point to this HTML page – either directly or using a ISAPI / NSAPI plugin which work directly on the web server.
This is the most optimal form of caching on the server. Serving such pages only require the same processing as a static HTML file. This is an ideal candidate for web sites which are accessible without login, or which have an “all or nothing” login – no user profile based access.

The next location is withing the web-app. The web-app is most popular location for caching contents. Most popular web platforms provide an option to cache parts of web page – either as a part of the language – like or JSP cache tag, or using third party components like oscache.
This is highly efficient as we are caching computed HTML – with the least possible amount of processing on the server end, while still allowing personalized / dynamic content for the rest of the page.
It is worth noting that such cache is not Clustered and here there will be an instance of the cached item on each web server instance. This will pose a challenge while refreshing the cache in case of event driven cache. This could also lead to in-consistent results for a short period in case of TTL caches ( two different users hitting different web servers could see different results).

So where is such a cache applicable? Well – almost everywhere. Lets say you have an application which requires authentication and personalizes parts of site – like navigation based on access rights. Here we can cache the “relatively” static parts of the page.

Third option will be to cache objects in memory. Almost all applications cache reference data in memory either on web-app or on the app layer or both. The cached objects are placed in some kind of singleton or static objects. The datastore is structured depending on the granularity of cache expiry – and also based on whether there are limited cache items or growing/very large number of items in the cache. Lets take a few examples to understand this better.
As in last article,
1) Departure boards of all London Airports. Here feeds come from different airports, number of airports is limited and all get flights for an airport get updated together. Here it might make sense to have one “hashtable” or similar per airport and flights as items within them. On receiving a new feed, the entire hashtable may be cleared. So here, cache clearing is done on retrieving a new feed and there is no worrying about cache running away with all available memory.
2) Flight results at or similar: Here the flight results depend on number and age of passengers, departure and return dates, preferences like economy, business class etc. It is likely that the same query may never get repeated – like If I searched for today’s flight, from tomorrow it will never be there. In such case, the cached result will just sit and eat away the memory. Such cases require cached items to be specifically recorded in a list – and the items in the list be cleared by a background thread- very much like garbage collection in managed .NET code or Java. The datastore in such case may be a static hashtable + a table containing object references and expiry times sorted by expiry time.
3) Inventory for an e-commerce shop. The Item summary including inventory may be cached at search results level. On navigating to item details, or on attempting to buy, the cached inventory for the specific item may be refreshed. The datastore in this case maybe something like a resultset.
4) Reference data: In case of any change, we may want to refresh all reference data elements.
The datastore in this case may be a static hashtable.

Remember that these object cache will be one per server instance ( 1 per JVM, 1 per App pool etc.)

The final option is to cache objects in database. Now there are two distinct scenario where we do that
1) We create materialized views and hence “cache” the table values in a way its optimal to retrieve.
2) Data fetched from external sources is stored in the database for subsequent use.

Cache is kept in database in multiple scenario like
a) No tolerance of stale data. Database cache is a single place and hence can be updated ( read observer pattern, state machines and related patterns) as soon as the source data gets updated.
b) Grid computing / 2 Tier application or a very large web-farm – in which case it might be sub-optimal to have the items cached at each node.
c) Data comes from external interfaces at a very high cost, and hence 1 fetch per JVM/CLR may be too in-efficient. Hence the item is cached in the database, and it may as well be cached in the individual machines.

In the next post on cache, I will explore in detail accessing and updating cache, especially event driven ones.

See my next post on Cache access and expiry


4 Responses to “Cache – Implementation”

  1. To get an idea of some of the cache types available in Java (and now in .NET :-) in Coherence, see:

    Basically, the idea is to trade off one type of resource for another. For example, if you keep more data in memory locally, it will cost more memory and it may cost more network (to keep it up to date), but it will save access time. On the other hand, if you have to go to a different server in the cluster to grab some data, it will add latency but may reduce overall network usage (less data to keep in sync).

    Every use case is different, so having the multiple options is important to being able to produce the optimal result.


  2. This is a superb post and may be one to be followed up to see how things go

    A companion e-mailed this link the other day and I’m excitedly anticipating your next page. Proceed on the top quality work.

  3. Pretty! This was an extremely wonderful article. Thanks
    for supplying these details.

  4. This excellent website definitely has all of the information I needed about this subject
    and didn’t know who to ask.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: