ObjectGrid offers developers more persistence choices

WebSphere XD ObjectGrid 6.0.1 is almost done now and will be shipping at the end of the month. It adds many features over the initial release and ObjectGrid should be competitive with other cache products from a function and price point of view for most customers.

I think we're starting to see products like ObjectGrid provide expand the persistence choices for application developers building applications. The database was the traditional persistence mechanism. Even HTTP sessions were initially stored in databases and many customers still do that.

As databases were the only choices, all data that was persistent was tagged as stored to disk. This can be an expensive way to store data that doesn't have such strict storage requirements.

HTTP Sessions are examples of this. Unfilled stock trades are another example, application state for a current user is another example. Products such as XD ObjectGrid provide application developers are more cost effective choice for storing data that needs to be reliable but not stored in disk.

Define reliable

The ObjectGrid now allows memory replication of data between one or more servers. This replication can be synchronous or asynchronous. Obviously asynchronous is faster but you may lose data if a failure occurs. Synchronous is slower but the data is guaranteed to be on the replica before it's committed locally. The ObjectGrid may not use a database at all in this case and would actually become the primary store for this data.

Application state can now be stored in replicated memory or an ObjectGrid when it makes sense to do this. This should offload the database signficantly depending on the database. It's cheaper because the ObjectGrid CPU licenses are significantly lower than a database and the ObjectGrid can scale horizontally using map partitioning across a large number of low cost servers.

All changes to an ObjectGrid are transactional and applications can register a transaction listener so that it can be informed whenever data is modified in the local ObjectGrid partition. Applications can use these events in a manner similar to database triggers. They can detect changes and then perform business logic in response to those changes. For example:

• Update data in another Map based on whats changed. Update a portfolio position when a stock price changes.
• Aggregate the changes in a Map. This could allow summary statistics for data to be kept/maintained as the data is modifed by the applications.

These listeners run inside the ObjectGrid servers and scale as the ObjectGrid cluster scales. This allows problems that previously required a large database to run much faster when the data is kept in an ObjectGrid.

Write through caches can accelerate batch jobs.

If data kept in a database can be staged in to an ObjectGrid cluster and then applications use the ObjectGrid to access/modify that data rather than using the database directly. This can signficantly offload the database as well as offer opportunities to access/manipulate that data in a more efficient/lower cost manner. A partitioned ObjectGrid can have application code run locally on each cluster JVM that operates on the subset of data stored within that JVM. Clearly, this code can run signficantly faster than a traditional database client that would have needed to access the database. The traditional approach would also put a significant load on a database and the database will become a bottleneck and this is expensive to correct, i.e. you need to buy a larger box with better disks. If the ObjectGrid is used then it's a much cheaper way to attain significant performance gains versus the traditional solutions.

We'll be seeing more capabilities being added to the ObjectGrid to better enable it's use in this fashion as we move forward over the next few months.

Comments

[John D. Heintz]

How does ObjectGrid in async mode differ from memcached? (http://www.danga.com/memcached/)

[Billy]

From what I can tell memcache is a non replicated partitioned remote cache. ObjectGrid can be configured to work exactly like this but it's only one way ObjectGrid can be configured. An ObjectGrid can also be configured to have a near cache which I don't see in the memcache pages but I'm restricted from downloading it without legal approval within IBM so my knowledge on this may be/is incomplete.

OG can also do checkpoint based recovery which can speed up bringing a fresh replica up to date and again, I don't know if memcache can do this.

OG is transactional and multiple changes can be placed in a single transaction and replicated atomically with fewer RPC than a non transactional approach and with atomic consistency. If you're doing multiple updates per request then this makes a big performance difference.

The other aspect is that ObjectGrid allows a cache server to be replicated synchronously or asynchronously, this doesn't seem to be supported by memcache and this can be a big deal for customers whose databases are not sized to deal with the normal load because they expected a warm cache in front :)

OG also supports cache preloading with recovery. This means that if two have a server pair acting as mirrors, the primary is preloading data. This is being replicated to the backup server online. If the primary fails before the preload has completed, the replica can continue from the last replicated point.

So, the big differentiator is that memcache is designed to be a remote partitioned cache but without the capability to run application code on servers and it can't really be used as persistent copy as it doesn't appear to be able to synchronously replicate. I think that ObjectGrid is as easy to use as memcache when ObjectGrid is used in this manner.

Lastly, memcache offers support for multiple languages which is cool and we don't do that out of the box as this point but products such as CodeMesh could be used to do the same thing more or less. Integration with LAMP is something I'm looking at though. There's a lot of it out there.

Again, I can't download it and compare so you're probably better able to judge than I can.

Thanks
Billy

[Alain Rogister]

Is it going to be available only as an option for WebSphere XD ? Or will it be licensed also for other editions of WebSphere ? Or possibly for other applications servers too ?

[Billy]

Everything, it's fully functional without WebSphere XD underneath it. ObjectGrid clients work on WAS 5.1 and up, ObjectGrid servers work on WAS 6.0.2 and up.

The ObjectGrid client should work on any JRE 1.3.1 or better, the server works on 1.4.2 or higher. It can be embedded in to other application servers or Java applications.

There is a much lower license fee when it's used in standalone mode. This is official (there's a PID with a seperate price) now whereas with XD 6.0, it was a special bid.

[Cameron Purdy]

If you look at the code for memcached, you will quickly understand how little it does, and how poorly it does it ..

However, for simple content caching where correctness, performance and availability are not terribly important, it's probably fine (and so was WebSphere dynacache).

Peace.

[Bill Higgins]

Wow. Cameron, you're definitely in the lead so far for smuggest blog comment of 2006!