Messaging only APIs need to go, JPA needs to step up

I'm getting tired of the need for separate messaging and data access APIs. You could characterize these APIs as:

• Ordered accessible data (OAD)

• Key accessible data (KAD)

Key accessible data arguably has the most capability and has seen the most innovation over the last 5 years. Most notably, the JPA specification has arrived which standardizes the POJO oriented work done in OR mappers like Hibernate and Toplink. We have automatic mapping, relationship management, query support, lots of nice stuff. Now lets look at JMS. It's about as stagnate as you can imagine. Why can't I go all the stuff with OAD as I can with KAD? Do we now need to enhance JMS. NO, NO, NO. We need to force messaging to simply extend the JPA like standards, not compete with them. Messaging is just state, state should be managed the same way whether its state from a message or state from an inmemory database like ObjectGrid or state from a relational database.

For me, this status quo isn't acceptable at all. The JPA specification is unfortunately very short sighted in this area and is too relational database focused. It should be a main specification focusing on entities and relationship between entities and a subspec specializing it to relational databases but a product that works with persistent state should have been able to use JPA APIs but alas, that isn't the case and the JPA 2.0 stuff doesn't look any different. Why couldn't we use the JPA spec for messaging or for non relational databases or as an API to an in memory database directly?

JPA type interfaces could easily support messaging semantics with one little change. OAD and KAD have different but not incompatible locking requirements. OAD needs a way to scan a filtered set of records  (think query) looking for unlocked records and then lock the first one and return it to the application. The JMS receive method does just this. If many threads/consumers go after the top of a queue then the locking model allows this behavior. Find the first unlocked message in FIFO order, lock it and return it.

Why then don't databases provide this locking model, scan for unlocked record and lock? I don't know. We have added it to ObjectGrid for the next release and this simple addition allows an ObjectGrid Map to be used for both a KAD and an OAD. This completely unifies data and persistent messaging and it's very, very cool.

No more is messaging data and key data treated differently, it's the same and it should be. Why can't I annotate a message with a relationship to data outside the message in a TABLE or ObjectGrid entity? It should be absolutely seamless and messages should not be orphans in your data model, a message is just another record type and it should be able to have relationships with other records and this should be supported first class.

I think it's been too long now for this change to happen in the industry. Messaging APIs are dying on the vine, ws-* stuff just redefines the existing APIs with ws-* APIs and transports but does nothing to simplify the life of a programmer. ws-* just provides a new syntatic sugar for the existing APIs and supports more protocols but fundamentally does nothing new for the programmer. We need one data programming model and that data programming model needs to support both KAD and OAD with the same data model, tooling and APIs. I say, split JPA in two specs and extend the base spec with an optional addition to add efficient OAD support to the API.

Comments

[Diego]

What do you think about SMILE project?
http://domino.research.ibm.com/comm/research.nsf/pages/r.distributed.innovation2.html

[Billy]

It's very cool. It's easy to imagine SMILE working with streams of versioned records from ObjectGrid data. We're adding support for SMILE to the next version of OG.

[Jags Ramnarayan]

Not sure if I understand your analysis to differentiate messaging vs data access API as primarily based on ordering (messagin) vs key based, but, it is interesting.
One could (and does in real life apps) define the key simply as a message header attribute and use filters in the subscriptions, no?

But, I do agree that a distributed cache (fabric, object grid, data grid, space, whatever) solution and aspects of messaging could be integrated. My perspective might be little different than yours - there is no need for an explicit messaging API; merely a "data fabric" API with some extensions to support subscriptions on a data model managed by the fabric. Much simpler, much more intuitive, and potentially faster.
Here is my view on reliable Pub Sub being an integral part of a data fabric -> http://jagslog.blogspot.com/
Perhaps, you might want to chime in.

regards
- Jags Ramnarayan

[Frank Kelly]

Yup - it must be in the air - I wrote pretty similar topics in a blog post of mine entitled "Java Wish List #1 - Broadcast JDBC data changes" (URL: http://softarc.blogspot.com/2007/03/java-wish-list-1-broadcast-jdbc-data.html) because I hit the issue pretty much everywhere I go - messaging and persistence are highly inter-related in most applications.

Best,

-Frank