Data Integration : You're Doing it Wrong

The cost of integrating information isn't cheap. Well, at least it isn't if you do it wrong. Chances are, you're doing it wrong.

The single most difficult aspect of data integration is the effort to achieve consensus. It isn't just that we are disagreeable people. It's also that it is a fantasy that there is a “common model” or a “global truth”. Different groups and individuals see the world differently and have different needs from information systems. Language, and therefore what we call things, isn't simply reflective of reality. It plays a constructive and interpretive role.

The problem is that our technologies force us to make choices about world views and pretend that things aren't changing constantly. This yields fragile systems and high impedance to change that cascades through our organizations. This translates to expensive, rigid and difficult to extend failure to give the business what they want.

Our friend Tim Berners-Lee and his Happy W3C Merrymakers have given us a set of technologies to help us solve these problems though. We forget that the Web he designed was not the public Web, but one to solve integration needs for complex organizations like CERN. The HTML bit that we have gotten so excited about is but a small part of the vision. We will introduce RDF and SPARQL as enabling technologies. They do not necessarily replace what you already have, but they do make it possible to share information with people you've never talked to: Collaboration without Coordination.