Semantics, Formats and Markets

I was reading Dare's response to Bill's post on format proliferation and RDF. Bill hopes that more folks will understand RDF and seems to see it somewhat as a silver bullet for the problem, while Dare is more pessimistic.

The way I see it, there are distinctly two issues here. The first is the obvious one, solving the n*m problem, where n = format count and m = format producer and consumer count. The idea behind any common format or format mapping / shaping / semantic extraction is to reduce it to n*1 + 1*m, i.e. n+m.

That's all well and good, but it has problems: lowest common denominator, semantic loss in conversion, inhibition of innovation (gated on lowest common denominator), barriers to entry (new producers want only to be concerned with their specifics, not a huge standard, while new consumers don't want to have to implement the world before being useful).

More germane to the RDF situtaion, a requirement to understand a meta-model as well as any given model itself is so large a barrier to non-specialists just trying to get their work done that it's highly unlikely to ever receive serious attention. More on that in a bit.

Official standards-body approaches are the other issue. Formats in an area of innovation act like a bubbling market; take-up of formats by consumers and producers determine the winner, until eventually network effects reduce the total number of formats to just a handful. The problem with trying to manage this market process via a committee is much like the problem of trying implement socialism: the total information embodied in various choices of one format over and above another, which a free market reveals naturally, is not available to committees. Committees tend to be dominated by large market players which have various strategic and political objectives that may be quite distinct, and indeed sometimes covertly opposed, to the average market desires for any given format.

The working programmer, at the end of the day, is either putting square pegs into square holes (in which case, no problem), or trying to put square pegs into round holes, and then having to create an adapter to convert square pegs into round pegs. An arbitrary selection of square or round as a "peg standard" doesn't necessarily help him for his specific needs; similarly, pointing at some generalized framework for describing the semantic meaning of square and round pegs respectively is far too abstract for him to get his job done efficiently - i.e. without investment whose cost exceeds the value of getting the original job done.