Community Development
I have finally gotten around to reading Clay Shirky's excellent book Here Comes Everybody
. I love Clay's writing style and the way his perspectives make me think. One of the points that really resonated with me was about open source. But before I get into it, I should say that when Clay talks about open source, he is really talking about community developed open source (not commercial or institutional open source). Readers of this blog know that the designation "open source" simply means that it is licensed under an OSI certified license. "Open source" says nothing about how the software was developed. Given that disclaimer, Clay nails it when he describes the advantage of community developed software:
The bulk of open source projects fail, and most of the remaining successes are quite modest. But does that mean the threat from open systems generally is overrated and the commercial software industry can breathe easy? Here the answer is no. Open source is a profound threat, not because the open source ecosystem is outsuccessing commercial efforts but because it is outfailing them. Because the open source ecosystem, and by extension open social ecosystems generally, rely on peer production, the work on those systems can be considerably more experimental at a considerable less cost, than any firm can afford. (from page 245 of the hard cover version)
This quote comes from a chapter called "Failure for Free" that discusses the importance of failure for innovation. The key point is that traditional companies can't afford to really innovate because they must limit their bets to initiatives that they feel have a high likelihood of success. Innovation winds up being constrained to small tweaks to things that are well known. Real breakthroughs are rare. Community development doesn't make better ideas but it reduces the cost (and risk) of trying ideas that at first seem unlikely to succeed. Out of all this experimentation comes unexpected successes. Even the failures provide useful data that can be turned into future successes.
When open source critics see the volume of incomplete or dead open source projects as evidence against the sustainability of open source, they are missing the point. A low success rate is critical to real innovation; but that is hard to understand by someone for whom failure is discouraged or not tolerated at all. It is widely accepted that the best design for an experiment is one with a 50% chance of failure: if you know what the outcome will be, the experiment is not worth doing. But if the cost (in time and resources) of the experiment is zero or close to it, the ideal failure rate is much higher.
While we have seen many examples of community developed software out-innovating its commercial peers, this phenomenon has very little to do with open source. An open source license can create an environment that invites contribution by reducing the risk that one will exploit and unduly profit from another's contribution. But what is more important is how the software is managed. Most successful open source projects have adopted an architecture consisting of a core that is tightly managed for stability and quality and is extensible by adding modules. The bar for module code is considerably lower than core code. This allows an open and dynamic community to experiment with new ideas. The community not only bears the cost of failure but it also brings in new perspectives that the core maintainers don't have. Some of these extensions will be absorbed into the core or will become part of a commonly used bundle. Others will wither and die.
This pattern of stable core and innovative fringe does not have to be unique to open source. It doesn't really matter how the core is licensed if entry into the fringe is open. The one area that open source licensing helps with the core is in achieving a level of ubiquity that attracts a community. High licensing fees or the bottleneck of a traditional software sales process can limit the size of the user base and discourage contribution. The community developer needs the promise of a large user base to justify the time investment of contributing his work. But open source is not the only way to achieve a large user base and there are plenty of examples of commercial software and SaaS user communities that have a successful code sharing ecosystem. One company that is particularly successful is Salesforce.com. Salesforce has created a very powerful API and module packaging framework. More importantly, they have priced the product to penetrate small-medium sized companies and non-profits. In particular, the Salesforce Foundation makes the product very inexpensive for non-profits. These strategies have infused the customer population with lots companies that are highly motivated to share. These customers are active both within the AppExchange community and also integrating 3rd party open source projects like Plone (see Plone/Salesforce Integration) and Drupal (see Salesforce module).
On the flip side, there are plenty of commercial open source software companies that not been able to leverage community development. There are two primary ways an open source software product can prevent community development from flourishing. First, there can be factors that limit adoption of the core like in the case of open core products that discourage the use of the open source version. Second, the architecture can be monolithic so the only way for an outsider to add a feature is to fork the codebase. All a software producer has to do is solve those two problems. The supplier doesn't even really need to provide tools for sharing code. If the install base is large (which usually means the software is useful) and the architecture is modular, developers will find a way to communicate and share code. They can use services like Google Code, GitHub, or SourceForge to collaborate. Too often companies put code sharing tools in place without solving those two problems and then complain when they don't see any activity. In many cases, the user audience is too small to support a contributing ecosystem. In other cases, the incentives are not lined up appropriately. Shirky calls the three ingredients of a collaborative community: promise, tools, and bargain. The promise is what the contributor can expect to get out of participating; the bargain is the terms and rules for participating. Open source licensing helps with the promise and the bargain. The open source promise is that others may help improve your code. The open source bargain is that people can't take exclusive ownership and profit from your work. Some communities, like Salesforce and Joomla! have thriving commercial extension marketplaces. In those cases the promise and bargain are different but they are very motivating.
Any widely used software application, if it is appropriately designed, can benefit from community development and the benefits are not limited to the successful contributions. Perhaps the biggest value of community development is increased innovation through reducing the cost of failure. In order to harvest this value, a company needs to actively monitor, analyze and participate in its community. There is a goldmine of information sitting in the chaff of failed contributions as well as the modules that do gain traction. Companies that ignore this value do so at their own peril.