/dev/websphere

"Penny wise and dollar foolish" applies to distributed caching too often

Many times I've had people compare the performance of WebSphere eXtreme Scale to a local hash map and comment that it's slower. Basically, they are comparing a remote hash map call to a local hash map call and yes, the remote one is much slower. But, this is an apples to oranges comparison. Try caching 600GB in the local hash map on every JVM requiring a cache? But, hey, it's faster, right?

Wrong. If you just need a local hash map then you should use one. A distributed cache like extreme scale brings expandible capacity, high availability of data in the cache, a network accessible remote very large cache, cache stays up when the application jvms are cycled and so on. It's different and thats why the fact that a local hash map is faster isn't really valid.

For a real world example, lets look at WebSphere Commerce Server. It can cache HTML page fragments in a local map (Dynacache) and in extreme scale also. If the cached data in present in both caches then the local map is faster but, that's not why we sell extreme scale to commerce customers.

An extreme scale cache is remote. You can cycle the commerce JVMs and when they restart, they are working with a hot cache straight away. This isn't the case with the local cache. It has to be regenerated and this costs of lot of CPU. This regeneration also hits the database during this time. So, we have extra costs of CPU, longer response times and database load if we cycle a commerce JVM and have to rewarm up its page cache. None of these problems exist with extreme scale as the page fragment cache because the cache contents isn't kept in the commerce JVM at all and therefore it survives a commerce JVM restart.

Now, imagine bringing down a box with 10 JVMs on it. With the local cache, the cost is 10x when you restart the box. With extreme scale, you has the cost of restarting the JVMs but no cache warmup costs at all as the cache survives the JVM/box cycle. It's ten times faster because those ten JVMs do not have to warm up their ten independent page fragment caches again or hit the database and the customers using that box see "normal" response times pretty much as soon as they are online versus waiting for the JVM caches to warm up in the conventional dynacache scenario.

Next, lets look at invalidation. If a large site had 100 JVMs and I update the price of an item, then all 100 JVMs need to regenerate the page fragments for that item every time this happens. They all burn CPU, the response time for serving that item is longer and there is database load. With extreme scale, the page fragments are invalidated but are only regenerated once, the first commerce server to request the page notices that it's not there, generates it once, puts it in the cache and now the other 99 JVMs have the new page even though they didn't generate it. This is a massive load reduction when prices or inventory levels are updated or promotions for product categories become active potentially impacting tens of thousands of skus in the system and the page fragments for those impacted skus. This means you can invalidate prices to match competitive prices in a more timely manner without worrying about high CPU spikes or database spikes. This is a major advantage for extreme scale with commerce server over the dynacache implementation.

Lastly, if each JVM has its own independent cache then potentially depending on which JVM a request is routed to, a different cached page fragment could be served, if pages are being invalidated aggressively then the site becomes a bit of a mess in terms of consistency of what users see across the cluster. WebSphere eXtreme Scale is a single logical cache shared by all commerce JVMs, they don't have a private local cache, they always go remote to extreme scale for page fragments. This means they are always consistent and a user will see the same cached page fragments no matter which JVM they are routed to.

So, when examining whether a customer should use websphere extreme scale with commerce server, it's important to focus on the massive response time, cpu savings and database savings benefits around edge cases, not steady state performance. The steady state CPU load of a commerce cluster with no JVM cycling, additions or data invalidations is likely to be a little higher with extreme scale versus a local dynacache. This is expected. If you have a completely static commerce site then dynacache with disk offload is probably a better solution. But, and I expect this is most customers, the content of the site changes and you want:

• users to see consistent page contents
• you want consistent response times if you need to ripple commerce JVMs or add JVMs
• large cpu and database load reductions when prices/inventory levels or promotions change
• more consistent response times means more satisfied customers versus a web site where depending on whats going on (edge cases), customers see widely variable response times.

Then extreme scale is the way to go, hands down.

Some reasons why WebSphere eXtreme Scale lowers Commerce Server CPU requirements

I've gotten quite a few questions on the just announced WebSphere eXtreme Scale integration with WebSphere Commerce Server. One of the top themes is around CPU usage.

Duplicate Page Rendering

This is the first major saving. A WXS cache is a separate tier and is shared between all the Commerce server JVMs. Once any commerce JVM renders a page fragment and stores it in the WXS tier then that fragment is immediately available to every other Commerce JVM. If there are N Commerce JVMs then that means the page fragment will be rendered once instead of N times. This is a significant source of CPU savings in the following circumstances:

• Commerce JVM cold start

Here when the commerce cluster starts initially, normally every JVM renders every fragment. With WXS, each fragment is generated once only, this allows the commerce cluster to get warm much faster than before with less CPU burned.

• Commerce JVM warm start

Here, when a Commerce JVM is cycled for maintenance or restarted after a period of being down, there is no need to rewarm its cache when using WXS. The cache is already warm when using WXS because the cache doesn't live within the Commerce JVM at all.

• Invalidation scenarios

Here, when fragments are invalidated, fragment only get rendered once instead of N times.

Cache management costs

Commerce Server using a private disk based cache normally. This cache is private that means the burden of managing that cache is borne totally by that Commerce JVM. The bigger the cache, the more management, the more garbage collection pauses, the more private disk I/O and so on. This cost is per JVM so N JVMs means N times this cost.

A WXS cache is not managed by the Commerce JVMs at all, the costs of managing the WXS cache is shared by the collection of JVMs hosting the grid. This removes this CPU cost from the Commerce cluster and makes that CPU available to do more useful things like sell products. In addition to this CPU saving, WXS scales linearly, no one JVM is responsible for the cache management tasks, the tasks are shared in a uniform manner between all the JVMs hosting the data. There is no single bottleneck. If you make the cache larger then add more JVMs to increase capacity, add CPU for cache request servicing and management and network. We can say that the cost of management for a single WXS JVM stays constant as the cache cluster grows in size. We move the cost of managing the cache away from the client application and in to a more scalable tier.

Summary

So, there are two principal CPU savings. 

• WXS eliminates redundant page fragment rendering in startup and invalidation scenarios. 
• WXS removes cache management costs from the client application tier (i.e. commerce) and makes them constant cost per cache tier JVM as the WXS tier grows.

If you have any other questions then please comment and I'll write up responses on the blog.

WebSphere Commerce Server now supports using WebSphere eXtreme Scale for page fragment caching

IBM WebSphere Commerce Server V7.0 FEP1 now supports using IBM WebSphere eXtreme Scale V7.0 ifix 3 for page fragment caching. This is a big deal for commerce customers. Commerce customers traditionally use dynacache with disk offload to store page fragments. Each Commerce JVM has it's own private page fragment cache and it overflows to a disk file associated with that JVM. This has worked well for many customers but it's time to dramatically improve this.

IBM allowed dynacache to switch to use a remote WXS grid instead of disk offload using a plugin starting with IBM WebSphere ND V6.1.0.25. This means that by adding a couple of lines to the cachespec.xml, applications that used the dynacache APIs can now store their cache in WXS instead of in the usual dynacache mechanisms. This allows these applications to:

• Shared a single cache between multiple ND JVMs. All JVMs see the same cache
• Handle very large caches. Applications can now comfortably cache 100's of gigabytes of data if needed in this shared cache.
• The cache survives ND JVM restarts. You can shutdown and restart a ND JVM and the cache is still hot.
• Items in the cache when invalidates are only refreshes once PER cluster, not once per JVM

These points are very important for Portal or Commerce applications that typically render expensive page fragments. Displaying page fragments from a cache rather than rendering them from sources every time can dramatically speed up an application with corresponding decreases in CPU costs and response times.

A normal disk offload dynacache can do this but had several disadvantages. First, the cache is private per JVM. This means that each JVM is wasting memory (heap memory and file system cache memory) caching the same data and wasting disk space. Each JVM also wastes CPU cycles updating fragments in the cache. The cache is private and every JVM must recreate invalidated page fragments. If there are N JVMs then each fragment is regenerated N times after an invalidation. If a JVM is stopped for a few minutes and then restarted, its fragment cache is stale and is usually cleared and then warmed up from scratch. This means users that are attached to this warming JVM will see poor response times during this interval. There can be hundreds of thousands if not millions of fragments in these caches so warmup times can generate significant disk I/O, application CPU and database load. All these negatively impact the whole cluster, especially the database load. The next issue is that each JVM has an independent cache and potentially this means that depending on which JVM a user attaches to, the user can see a different version of the page fragments. They are not shared.

Most of the benefits of using WXS with WCS/ND/Portal comes from these simple facts:

• The cache data is stored independently from the application JVMs.
• The cache capacity can be very large, 100s of GBs or bigger without requiring disk.
• It's a shared cache, every application JVM sees the same cached fragments, guaranteed, regardless of how long the application JVM has been running.
• When cache data is invalidated, it is regenerated ONCE no matter how many application JVMs are using the cache. Invalidation now has constant cost.
• The cache runs from network attached shared memory (the grid), not disk. It trades disk I/O for network bandwidth.

These facts are a major advantage to WCS, Portal and ND customers who use dynacache for page fragment caching today.

The fact that the cache is now in a separate tier means that you can restart application JVMs and not suffer from cold cache syndrome. The restarted JVM will run with a hot cache immediately when it restarts. You don't need to warm up the JVMs cache. Customers using a newly started JVM will instantly see the same response times as other longer running JVMs are seeing. There is no warm up time as far as the cache goes.

Disk offload was good for when large retailers wanted to store lots of page fragments, typically < 20GB, and there isn't enough memory to have a cache that large PER JVM. However, WXS allows those customers to keep using 32 bit JVMs with good GC pause times and still cache very large amounts of data remotely. The thing to remember here is that you need to cache say 5GB of data then with replication, thats 10GB of data. If you have 10 application JVMs then normally you'd need 50GB of memory total with a conventional cache. When using a remote WXS cache, you only need a total of 5GB with no replication or a total of 10GB with replication no matter how many application JVMs you use. Thats a significant saving on memory. 50GB versus 5GB, a 10x saving with no replication, and 5x with replication. This is a major advantage for customers running applications on virtualization environments where memory is a premium.

The cache is shared between all application JVMs. This means that no matter which application JVM serves a web page, the page will be identical to pages served from other application JVMs. The cache is NEVER stale. Every application JVM is guaranteed to see the same page fragment in the shared cache ALWAYS. This gives a much better customer experience especially during periods where the fragments change frequently such as black Friday or cyber Monday for retailers.

The cache is shared and this means that when a page fragment is invalidated then the next application JVM to look up that page will recreate the fragment and store it in the cache. Once that happens all other application JVMs will now find the new fragment straight away. This means you only recreate a fragment once per cluster, not once per JVM. This is a major CPU saving that increases the more application JVMs you use. Invalidating a page fragment now has constant cost, no matter how big the application JVM cluster is.

A cache that spills to disk requires fast disks. A WXS cache runs out of memory. Memory is faster than disk, even SANs. Customers could look at using a flash or solid state disk for this but you still don't get around the problem that it's a private cache per JVM. You won't get the advantages listed above with a solid state disk.

So, this all sounds too good to be true. Whats the catch? Very little actually. Network bandwidth between the application tier and the cache tier is the main catch. WXS compresses the data in the cache tier and across the network. This gives a saving of around 2.5 or 3:1. If possible, I'd recommend using a separate gigabit network port/subnet for the communication between the application tier and the WXS tier.

The advantages here are significant over the older technology. You can switch to this by basically editing the cachespec.xml file and starting a remote WXS tier. It's very low risk from that point of view, you don't need to change your application, you just edit the cachespec.xml.  If you use dynacache for page fragment caching right now then I think you need to look at this technology. It's relatively easy to try it out and the savings/advantages are significant.

It's important to realize that the bigger your application tier cluster is, the bigger the savings here. It's also easy to scale out the WXS tier by simply adding more JVMs to add more CPU/network/memory to the cache tier. WXS scales absolutely linearly as you do this.