CTO Articles
Published in IT World
October 31, 2006
Interoperability as a state of mind
In my line of work I regularly have to deal with issues related to the interoperability of file formats. Information in format A needs to be converted to format B because application A is being replaced by application B ... That sort of thing.
In such scenarios, the first question invariably relates to file format A. Is it open or closed? Is it binary or textual? Is there an API? What coverage is provided by the API? That sort of thing. In earlier years, I used to read too much into the existence of file format documentation and the existence of plain text formats. It got me into trouble sometimes.
You see, the issue is that documentation, textual file formats, full coverage APIs and so on all help when you need to move data from one application to another, but none of them are guaranteed to. The truth of the matter is that interoperability cannot be reduced to a question of documentation or use of ASCII or anything of that nature. True interoperability is not a technical thing, it is a human thing. Interoperability is a state of mind.
Put yourself in the mind of a fairly inexperienced programmer (if you are not there already). You have data in memory and you must store it to disk in order to read it back in at some future time. What is the easiest thing you could possibly do? Well, that would be to simply dump the internal data structure out to disk byte for byte. An operation variously known as dumping, serializing and pickling. This will achieve the primary objective. Now, could you do this using just ASCII and therefore claim to have an interoperable file format? Yes you could but it would not be a helpful thing to claim. The ASCII-ness of the data doesn't take away from the fact that the data is represented in a way that makes it useful for your application but a lot less useful to other applications that might want to read the data.
Put yourself in the mind of a more experienced programmer (if you are not there already). You know from bitter experience that simply dumping your internal data formats out to disk is troublesome. Whenever your application changes, even slightly, the data cannot be read back in without the aid of some sort of conversion. As an exercise in defensive programming, you decide to make your data on disk less reliant on the precise details of the internal workings of your application right now. Time spent making the data more robust on disk will pay itself back a hundred fold come version 2 and version 3 of your application.
So far, we are operating within the confines of one programming worrying about one application. A degree of interoperability makes tactical sense here. Is there a logical progression from here to an even more interoperable file format designed to be read/written not only by differing versions of the originating application but also differing versions of completely different applications? Is it reasonable to say that good, defensive programming tactics tend to lead developers towards creating interoperable file formats?
No. That is a step too far. There is a world of a difference between protecting your own back and protecting the backs of others. ASCII or no ASCII, XML or no XML, a truly interoperable file format is one that has been designed specifically for that purpose, one that organically grows out of internal defensive programming practices.
Interoperability is a state of mind. You need to be thinking about the other guy to do it right. Only people whose job description includes thinking about the other guy, have the luxury of doing so.
It is worth keeping this in mind the next time you encounter an internally developed file format which overnight appears to have become an interoperable file format.