Plugging Plugins: On the Ignorance of Crowds

This is the first of an occasional series of essays on the notion of software Plugins, also known as extensions. Plugins are a very powerful programming model, as I will try to demonstrate. Hence the title “Plugging Plugins.”

I first thought of this a few months back after hearing a talk by my colleague Sean Dague on the problems of starting an open-source project, during which he mentioned that using a plugin architecture made a project more accessible and so lowered the barrier to entry in attracting new developers. Expanding on that should have been the first of these essays, but I recently came across an article by Nicholas Carr that prompted me to write this post to kick things off.

Carr recentlywrote an essay The Ignorance of Crowds that begins by noting this is the tenth anniversay of Eric Raymond’s seminal essay The Cathedral and the Bazaar, and goes on to argue that the “bazaar” approach of peer production, especially as it applies to open-source software, is limited and that successful projects rely on the “cathedral” approach of a small tightly-coupled team.

Carr’s essay is well worth a read. The concluding paragraphs summarize his doubts about the power of the bazaar approach:

But if peer production is a good way to mine the raw material for innovation, it doesn’t seem well suited to shaping that material into a final product. That’s a task that is still best done in the closed quarters of a cathedral, where a relatively small and formally organized group of talented professionals can collaborate closely in perfecting the fit and finish of a product. Involving a crowd in this work won’t speed it up; it will just bring delays and confusion.

The open source model is also unlikely to produce the original ideas that inspire and guide the greatest innovation efforts. That remains the realm of the individual. Raymond was clear on that point when, toward the end of his paper, he examined some of the “necessary preconditions” for the bazaar model of production. “It’s fairly clear,” he wrote, “that one cannot code from the ground up in bazaar style. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode.” In a recent e-mail to me, he was even blunter. “The individual wizard,” he wrote, “is where successful bazaar projects generally start.”

Matt Asay, a software executive with long experience in the open source movement, agrees. “All open source projects — without exception — are started by one or two people and…have a core development group of fewer than 15 developers,” he says. “The most you can hope for [from the broader set of contributors] is bug fixes.” Asay warns that trying to expand the core decision-making group to include more of the “community” can backfire, as the resulting decision-by-committee approach tends to produce “stale, conservative code.” In other words, keep the bazaar out of the cathedral.

So if you’re looking to bolster your company’s creativity, you should by all means look for opportunities to harness the power of the crowd. Just don’t expect the masses to take the place of the lone wizard or the band of mages. The greatest breakthroughs will always begin, to quote Eric Raymond once more, with “one good idea in one person’s head,” and the greatest products will always reach perfection through the concerted efforts of a highly skilled team.

Note in particular Carr’s statement that “Just don’t expect the masses to take the place of the lone wizard or the band of mages” and Matt Asay’s comment that “The most that you hope for from a broader set of contributors is bug fixes.”

Though it is certainly true that all open-source projects begin as the work of a small group, it does not follow successful projects must therefore use the cathedral approach. You don’t need to rely on Carr’s “mages” or magicians. You can use the magic of plugins.

A plugin architecture lets you move beyond a reliance on a small team to coordinate all the work. Yes, you need a small team to define the core piece of the software, but if you have a plugin architecture then others can use it without having to coordinate their work with that small core team. They can extend the software on their own, constrained only by their imagination and programming skills.

Consider for example Eclipse and WordPress.

Eclipse started as an IBM-intiated effort to build an ” open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle.” Plugins are fundamental to the Eclipse architecture, and I think are key to the subsequent success of the project, which has evolved to an independent organization that is the keystone of an ecosystem consisting of scores of companies and hundreds (thousands?) of plugins, most of which are commercial applications built on the open-source base.

WordPress started as a small PHP application for supporting blogging that also supports plugins, and it has grown, thanks in part to the many plugins provided by independent developers, to the point where hosts over a million blogs, and the software is also used to power many other blogs at other sites, as well as to create complete web sites and not just to support blogging.

Plugins support a hybrid model of production, using the cathedral for the core and the bazaar for developing plugins. Plugins allow the creation of groups that can use their own peer-production model independent of the core group, and in so doing can allow a project to scale far beyond what a small group can accomplish.

The model can even be extended to allow scaling of groups of related projects. Consider for example, LAMP – Linux, Apache, MySQL, and PHP. Though it is commonly viewed as a programming model, it can also be viewed as an extended plugin architecture, one that combines the efforts of four separate projects, each maintained by its own core team, to produce a pluggable-architecture base that has been used to create many of the web sites we use every day.

I’ll explore other aspects of plugins in future posts as time permits, so we can plug along together.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

  • Pages

  • June 2007
    M T W T F S S
  • RSS The Wayward Word Press

  • Recent Comments

    daveshields on SPITBOL for OSX is now av…
    Russ Urquhart on SPITBOL for OSX is now av…
    Sahana’s Respo… on A brief history of Sahana by S…
    Sahana’s Respo… on A brief history of Sahana by S…
    James Murray on On being the maintainer, sole…
  • Archives

  • Blog Stats

  • Top Posts

  • Top Rated

  • Recent Posts

  • Archives

  • Top Rated

  • %d bloggers like this: