I’m impressed with the API. I have it working now and we’re in the process of testing it internally. The current setup uses all public APIs of IBM WebSphere eXtreme Scale and persists to a partitioned replicated grid backed by a DB2 database. I’m currently planning to ship it as a sample with full source code for WebSphere eXtreme Scale V6.1.0.5 very soon. Write behind logic is used to take the database out of the picture from a performance point of view for most use cases.
The database uses a fixed schema and I’m using IBM purequery as the interface to the database so it’s pretty fast and lightweight especially with the write behind logic turned on for batching.
The API layer is extremely similar to Redis except for the following restrictions/additions. These restrictions are easily removed with a small amount of coding for the extra types that are desired. We'll probably add support for Boolean,byte[],Double and Date before shipping the sample.
- 1) Keys can be String or Long
- 2) Values can be String or Long
- 3) Lists and Sets must have String keys
- 4) Lists and Sets only have Long values
- 5) Get operations can use a near cache if desired.
The API is very easy to use. I’ve been showing it around and the best comment so far is that it looks very grailsy and that’s pretty accurate. Everyone likes it that’s seen it.
From an ease of use point of view or just to hack something together then it’s pretty cool. Very easy to use and explain to someone and you can get productive with it very fast.
Installing it is straight forward. Get DB2, make a database, add 8 tables. Get Purequery jars. Get IBM WebSphere eXtreme Scale, link the library on the classpath, start servers using the supplied xml configuration files. Then use the client library (Java only) in your web application or console application and just code away. Purequery apparently works with other databases besides DB2 so it's not limited to DB2 but it'll definitely go faster with it due to the non JDBC layer used by purequery to talk with DB2.
It scales out nicely. The entries are dynamically partitioned across the set of live server JVMs and IBM WebSphere Scale transparently scales everything out if you start additional JVMs while it’s running. The extra JVMs allow more CPUs/network and memory for servicing requests on the entries. You can configure the level of fault tolerance that you want. Synchronous replication or asynchronous replication. A modest server should be able to handle tens of thousands of requests/sec with very low latency (single digit milliseconds).
Implementation wise, it’s been interesting. The highlights of the implementation so far would be writing a generic pure query POJO loader and using the grid as a 2 layer persistence store to be able to manage the list and set entries. This 2 layer approach was necessary because the application wants to store simple lists and sets as entry values. Persisting this to the database directly wasn’t efficient as the Loader doesn’t know which elements are added/removed to the list/set. The solution was to use multiple maps with agents. The client API library uses Agents to implement the List and Set primitives and they use 3 maps for a list or set. Map 1 has a string key and the value is a List/Set of Longs. This map is queried using normal Map.get verbs on the client to get the current list if needed. This allows the near cache to be used for the list if required. Map 2 is a database backed Map for the list/set header. Map 3 is a database backed Map for the List items or Set items. The agents job is to keep all three maps in sync. Write behind is used for Map 2 and Map 3 to get the database out of the critical path for updates. It sounds a little complicated but it's not a lot of code here.
The database schema is key/value tables (two columns) for String/String, String/Long, Long/String and Long/Long key/value storage and then ListHeadStringLong, ListItemStringLong, SetHeadStringLong and finally SetItemStringLong. So 8 tables in total. I wrote the Loaders and APIs using generics so it’s very easy to add say a value type for Date or Double for example. Examples of these are in the code but I haven’t made the extra tables. Date is easily replaced with the Long support. Dates are just longs anyway counting milliseconds since 1970.
The biggest issue/advantage with it is that the application orients towards a key/column or column oriented style. Each entity, for example: user, ends up having a key/value pair for each attribute. User might have 3 attributes:
- 1) Long/String for username
- 2) Long/DOB for date of birth
- 3) Long/String for password
- 4) Long/String for bio
This means the memory used is higher than if a POJO was used containing all 4 attributes. But, if we did this then we’d need a BLOB value or we’d need OR mapping which makes it more ‘clumsy’ for want of a better word. I tried trying to add structure but it make it less turnkey and harder to get going so the column oriented approach is what it's using.
The column oriented approach is very easy to use but is not schema oriented. It makes it very easy to add attributes on the fly without restarting the application or grid. This is attractive when prototyping or working with a production application that’s in flux and getting developed at a fast pace.
Reports against the database tables are not easy. The schema doesn’t make sense for most reporting tools. You’d need an export to CSV or another table to do it. But, the trade of here is that it’s easy to develop with despite all this and that’s what makes it very cool to prototype with.
Sharing a grid between several applications is pretty easy also. You can imagine using a prefix on all the keys for a certain application, effectively isolating it for the others.
Overall, I like this a lot. It’s quite powerful and a lot of simple applications could use an API like this and get productive with it rapidly. The fixed database schema and column oriented approach makes it a rapid development system and it’s very flexible/extensible at runtime. WebSphere eXtreme Scale makes the whole thing scale very well as it runs an elastic partitioned in memory data grid in front of the database all but removing the database from the performance picture completely. The ability to turn on the near cache for gets means that in our sample applications, remote calls to the grid are only needed for set/remove and list/set operations which are usually much rarer than get operations.
I think once the sample is published, I can see quite a lot of projects using this API instead of the normal one simply because of the column oriented style advantages and turnkey operation.
