I was explaining XTP in a meeting this morning and covered the usual aspects. It (in the form employed by ObjectGrid and its gigoherence competitors) uses replicated memory based storage for persistent state AND it uses a self healing/scaling grid to deploy that storage fabric on.
Someone asked if XTP is just about memory based replication then? I said no. If it was just about memory based replication then products like IBMs SolidDB or Oracles Timesten would be satisfying all XTP application requirements today and clearly neither product is really an XTP product. They accelerate an application needing a single database instance by eliminating disk I/O as a performance bottleneck but it still won't scale out or heal itself. If you use more data than will fit in memory then they aren't any better than a conventional database like UDB unless embeddedness (is that a word even?) or footprint is a factor.
So, if XTP isn't about memory based replication then what is it about? Memory replication is a single machine accelerator technology that eliminates disk I/O as a factor by using another machines resources to make an in memory copy of critical data on the main box.
An XTP platform isn't really about accelerating single box throughput, it's about organizing a set of boxes to work together to allow the power of all boxes to be harnessed to collectively run the XTP application. It automatically distributes ownership of the data (using some partitioning scheme) to the current set of servers and redistributes ownership of the data when boxes are added or fail. It automatically handles replicating the data from box to box and handles the availability of singletons running in the application that are tied to that data.
Those servers could be using a conventional database that is being controlled/managed by the XTP platform. A fictional example here would be if ObjectGrid managed DB2 instances instead of its own data containers. Each box in the grid would then have a database instance. ObjectGrid could create a database for each partition, assign ownership of a partition database primaries and replicas to a pair of database servers dynamically and then repeat for all partitions. ObjectGrid would basically be creating a primary database and a replica database on a pair of boxes and then instructing DB2 to create a replication link between them. ObjectGrid does this today for it's data containers and partition primaries and replicas. This would allow us to build a grid of 1000 boxes running a thousand DB2 databases with each one hosting a fraction of the total databases and each one acting as a backup for others in the grid. If a new box is added then we'd migrate some fraction of the partition primaries or replicas to the new box.
The extreme aspect here isn't single box performance, it's making the large grid manageable and avoiding the disk infrastructure (shared SAN drives, NAS etc). We're scaling up the grid as boxes are added with no centralized resources. Each box adds more CPU, memory, network and disk to the equation. There is no central resource that is bottlenecking how fast this grid can process transactions. The customer doesn't have to manage the data layout, redistribution when new boxes start (i.e. auto scale out), failure recovery and failure compensation (i.e. add a replica to bring us back to a pair when a box dies). The platform is self organizing a bunch of single box things in to a grid and making it even possible. It's enabling the extreme part. Now, we could get a very high performance application running collocated with DB2 using this setup. Replace DB2 with Solid DB and you have a memory based alternative but it's still extreme. The memory based one with Solid DB offers a higher throughput per box (no disk) BUT is limited to what fits in memory where as the DB2 one can store much more data because it's disk based. So, it's a TPS rate per box versus capacity per box discussion.
Clearly, ObjectGrid currently uses its non SQL based Java engine to host data in the same kind of infrastructure. But, the extreme part here isn't the memory based anything, it's the infrastructure for enabling the grid and organizing the data/application layout on it. So, XTP isn't about memory based anything, it's about automatically enabling this kind of scale out on a shared nothing architecture and possible collocation of business logic with that data to eliminate more path and network hops thus improving latency.
Sounds ultra high-tach, yet interesting info, thanks
Posted by: Dropshippers | July 25, 2009 at 09:59 PM