Saturday, October 18, 2008

The Search for Differentiation and Build vs Buy/Adopt/Coopt

There's been some chatter in the development blogosphere recently about that old chestnut, the build vs buy decision; but in these days of open source development, there are more alternatives to build: adoption and cooption (private forking / customized modules etc.).

Jeff Atwood wrote an article about not being ashamed to write your own software (i.e. build) when the functionality in question is core to your business. Much of the reaction to Jeff's fairly strongly stated article has been pulling in the opposite direction. I'd like to argue back in Jeff's direction, though.

The basic trouble I have is that the vast majority of open source is crap. Once you tread outside the critical path of high-user-base server-side software, like (to take three examples completely at random) Linux, Hibernate and MySQL, you very quickly get stuck in the mire; even then, much of the "good" open-source software is only good in parts.

For example, if you're looking for a mature, well-designed and stable interface for writing drivers for Linux - exactly what is needed to make supplying binary drivers viable for hardware vendors - tough luck, buddy. Economically speaking, incentives in open source are almost always not aligned with consumers' interests, but rather with the companies which hire the developers: and those guys are usually consultant / service shops, i.e. aligned with corporate customers, not consumers. Firefox is an exception because of Google advertising revenue, which is correlated with consumer adoption, thus their incentives are aligned.

Rails has a solid concept and has a reasonably good API, but its execution is, IMHO, not great. I have little patience for server-side technologies that aren't thread-safe, as request/response cycles are trivially parallel; though, from an historical viewpoint I can understand it, as relatively few scripting languages implement threading. All the same, though, the kind of coding that would make a server-side library thread-unsafe, I wouldn't tolerate even if the platform was single-threaded: it implies poorly-factored mutable global state. I wouldn't touch PHP with a barge pole - I have zero technical respect for the language.

To compete, you need differentiation: you need to do something, for some subset of the market, in a way that is head and shoulders better than every other choice. Open source (or, indeed, buying) can only give you what everyone else has; you need to add in the magic yourself that makes you stand out. Private forks of open-source software are one way of doing that, but is vulnerable to all the usual problems that forking has (staying current, merging bugfixes, divergent evolution, etc.). Creating your differentiating software yourself means your codebase will be smaller and more perfectly adapted to your problem domain, and thus easier to maintain and keep up to date than a private fork of an ongoing public open-source project.

The only real question is which software in your organization is, in the language of strategic management, going to help give you a sustainable competitive advantage. Open source software and third-party libraries can never give you that (unless you own the founding team & brand), because they are equally available to your competitors. Choosing the balance of build vs buy means trading off risk and cost versus lack of differentiation. You have to measure and compare the resources available to you and the risk of non-completion of differentiated software, taking into account opportunity cost, with the lack of differentiation that would come from the buy / ad/coopt decision.

Where the balance lies thus depends on both sides: how much [human+monetary] capital you have, versus the competitive landscape on the other. Whether a third-party library exists or not, in and of itself, is not a sufficient reason to always use it. Making users happy isn't sufficient; you must make them happier than the competition, and that's not going to happen if you don't do something the competition isn't already doing. The only thing that holds you back is risk & capital versus the reward in differentiation.


Lars Fosdal said...

An observation I have made over the years is that the more popular an OpenSource library becomes, the more functionality people try to graft onto it.

What once was lean and mean, bloats out because more people are adding stuff that wasn't in the original design, and some might have an different idea about what the software really should be capable of doing.

The difference between this kind of constantly morphing mindshare code, and meticulously homecrafted code where there only was a handful of people involved and where there were a good understanding of purpose and limit in scope, is significant.

In the end, people abandon the now bloated mish-mash of code, and move on to a newer, fresher, leaner lib. Then the cycle repeats...

OpenSource (and commercial libs too) could use some stricter management and stronger focus to keep it simple.

Anonymous said...

I read Jeff's article too, and in general I'll agree with him.
You need something to differentiate you from the competition. And using open source that might not be directed towards your particular problem domain, can ultimately eat up a lot of man hours trying to make the open source fit your business.
The bottom line is, like Jeff wrote:
The core business of your company should be based on your own source! Period.