Daily Archives: December 4, 2006

Yogi Yarns – On being lucky. First is not always best

Yogi Berra:

“I always thought that record would stand until it was broken.”

I hold a record. It won’t ever be broken.

I can honestly say I am the co-author of IBM’s first open-source project, a Java source-to-bytecode compiler called Jikes.

I’m the guy who got IBM to write its first open-source license. I’m hoping it entitles me to a free round in any open-source bar in the world, if there is such a bar.

But I’m also very lucky. That I can say I was first doesn’t mean I was best. I was just … lucky.

I was lucky in several ways.

First, and most important, most of the code in Jikes was written by Philippe Charles. He wrote about 80 per cent of the code, the most difficult part. I wrote the byte-code generator and much of the infrastructure, but all the credit for the code quality in Jikes should go to him.

His work is a remarkable accomplishement. His design has stood the test of time. No one, and no corporation, has come close to matching what he accomplished. Jikes shows outstanding design. And that’s not counting the Jikes Parser Generator, JikesPG, which is solely Philippe’s work, and is I think still the best parser generator yet produced.

I can also claim some credit. Though I may not be the best programmer, I am persistent — damn persistent. I do try to see things through until the end. I did see the opportunity to take Jikes open-source and I saw that through to the end.

Second, I set out on the path that resulted in Jikes going open-source in early June 1998. I had no idea that others in IBM were on a path that would result in IBM’s engaging with Apache to make their httpd server IBM’s web server.

That was my great good fortune. If I had started earlier, no one would have thought it important that IBM release Jikes as open-source. If I had started later, others would have beat me to the finish line.

Indeed, Jikes was first only by a week or so. Less than ten days after the Jikes release IBM released the code for postfix, a work that I think all would agree is much better than Jikes.

Postfix is the work of a single individual, Wietse Zweitze Venema. It is true world-class code, as good as it gets. As it happens, Wietse, Philippe and I worked in the same building, IBM’s Hawthorne Lab, though we never met each other, and I only learned of the release of postfix in the press, as did the rest of the world.

By the way, there is a hilarious story about the postfix release, first told to me by Peter Capek, and related in part in Andrew Leonard’s incomplete history of open-source that can be found at (link TBSL).

The story goes as follows. Jikes went out on December 7th 1998. Postfix went out eight days later. Though Jikes was mentioned in the trade press, the postfix release was announced in an article in the New York Times Business section that had a headline of the form, “IBM releases free e-mail program.”

IBM’s CEO in those days was Lou Gerstner. He saw the article in the NY Times and, having paid about $3.5 billion for Lotus just a couple of years earlier, wondered what was afoot with IBM’s giving away a free e-mail program.

So he called IBM Research. As it happens, the key executive involved was away that day, so Lou then called the head of IBM Research, and asked about the postfix release. He was told, “I never heard about postfix. What’s a postfix?” So he then called others, including the head of IBM’s software group, and got the same response, “What’s a postfix?”

Lou then blew a gasket, and told all involved to cease and desist in any activity in this area called “open-source” until he had been briefed fully on just what it meant.

As it happens, while I was not directly involved in the postfix story, I have direct knowledge of the immediate past before it, and what happened later.

Looking back, I think that IBM’s serious enagement in open-source, beyond the initial Apache engagement, can be traced back to a single meeting in a room in Toronto in early October 1998, at the annual meeting of the IBM Academy of Technology, a group of 300 or so of IBM’s most senior technical employees.

I was not a member of the Academy, but was invited to present my experience with Jikes as part of a meeting on open-source called by one of the Academy members. I find it noteworthy that the member was a woman, Susan Malaika. She had attended a number of meetings in Silicon Valley earlier that year, and had often found herself one of the few women present, and I am sure that none of the male programmers who spoke to her had any idea she was one of the world experts in relational database technology.

Part of the Academy’s job is to look into the future and see what technologies are coming down the pike, in particular what are called “earthquake technologies.” These are the technologies that have the potential to reshape the industry. For example, if was an Academy member named Mike Cowlishaw who, I was told, went to Armonk in early 1995 to alert senior IBM management about a new technology called “Java.”

Susan and others thought open-source might be such a technology, and so called the meeting.

As a guest of the Academy I had the unique pleasure of hearing Lou’s presentation to the members in an hour-long presentation.

It was a tour-de-force presentation. Lou was proud, very proud, that he had been elected to the American Academy of Engineering, and he was masterful — talking to the folks he respected most, on their terms and on their turf.

At the end of his presentation, Susan asked him about “open source.” It was clear from his response that he knew of the term but that he had not yet seriously engaged with it.

As I mentioned earlier, Lou shut down all open-source activities in the middle of December 1998.

I was invited to participate as part of the IBM presence in the first ever Linuxworld, in San Jose in early March of 1999.

I recall the night before the conference started as clearly as anything else in my IBM career. That’s because it began with a presentation by the executive, Robert LeBlanc, who had led the study Lou had requested.

Whenever I have described Robert’s presentation, to IBM colleagues or at this moment as I write this, his words still send chills down my spine.

That’s because Robert said that Lou had approved IBM’s engaging in open-source as a serious business issue, a very serious business issue, way beyond anything I would have imagined had I not heard it spoken so directly. That a corporation the size of IBM could make such a dramatic strategic change was breathtaking.

It was also an occasion to make me proud to work for IBM. Indeed, one of the great joys of working for IBM is that such occasions come more often than one would expect.

Such occasions are also an invitation to blog, as I will discuss in a forthcoming post.

Yogi Yarns – IBM throws in the towel

Yogi Berra:

“The towels were so thick there I could hardly close my suitcase.”

We continue our occasional series on open-source and “forking” with the tale of how IBM first engaged with open-source as a serious business issue, why it “threw in the towel” and took another path.

I first learned this story from James Barry. He is, I believe, the unsung hero of IBM’s engagement with open-source. He left IBM not long after the events I will relate below, first to work with Brian Behlendorf at collab.net,then later to Jabber, and then returned East when his beloved wife decided she preferred to live on this coast.

Back in early 1998 James was the product manager for what is known as “websphere.” Indeed, I think he may have helped picked that name. The issue he had to face was how to market what we today call a web server.

Back in those days IBM had its own offering, I think it was called Domino server. James faced a grim reality. Domino server was used by about 50,000 sites while a piece of software with the same function from Apache was used by about 500,000 sites, about ten times more.

James’s great insight was two-fold:

  • No amount of marketing dollars and skills, however well applied, gave IBM a realistic chance of competing with, yet alone displacing, Apache’s server.
  • In particular, web server technology has a unique property. Once a company decides on a web server, it becomes a key piece of their corporate infrastructure. To change it is not a task to be taken lightly. It’s as if a plumber were you to offer to replace every sink if your house with a fixture with gold faucets, so long as you agreed to replace each and every pipe in the house as part of the installation.

So James and his colleagues embarked on a great adventure — to convince IBM to jettison Domino and engage with Apache.

As best as I can tell, James spent most of the time over a period of many months on a plane, with occasional landings at various IBM sites to sell this vision.

There are a few URL’s I need to include here, and will do so at a later time:

  • Forbes article from 1998, “We’re doing a deal with a web site?”
  • Andrew Leonard’s Salon series on the history of open-source
  • An interview with Brian, but I forget where it came from.

(I’ll try to track them down as time permits, but will continue this post in the interimn rough form.)

In any event, IBM engaged with Apache in June 1998, and the rest is history.

A notable aspect of that history is that IBM help create the Apache Software Foundation. IBM didn’t want to “do a deal with a web site” and so helped establish a formal organization, what is now known as the Apache Software Foundation, in order to have an entity to deal with.

What is also worth noting is that Apache asked that IBM provide meaningful contributions to Apache,and as best I can recall, IBM found it challenging to find programmers skilled enough to compete with Apache’s programmers on their home course. This is yet another reminder that the best open-source programmers are truly as good as it gets.

But the real lesson to be drawn from this experience is that corporations, or at least well-managed corporations, don’t engage in open-source for the hell of it, or on a whim, or for the fun of it.

Corporations enage with open-source when it makes good business sense. No more. No less.

And I equally think that those folks who consider themselves members of the open-source community, not bound by any corporation, should also take the same view. Engage with corporations in open-source activities only when it makes sense for the community.

We are, after all is said and done, or in the words of corporate-speak, “at the end of day,” in this joint enterprise together.

It is the community that provides the core technology and defines the rules of the game, and it is the corporations who play by the rules who provide resources that enable the growth and maturation of what I refer to as the “open-source artifact,” the collections of thousands of packages that have brought us to where we are today.

For either side to treat the other with less respect than is due is to demean the whole enterprise. We all deserve better than that, and each of us should do our best to make it so.

There is also another side to this story. As if often the case, a primary story or even can have interesting side effects. This was certainly the case with IBM’s engagement with Apache, as it happened at just the right time for me. It was that event that made Jikes possible, as I will relate in the next post.

Yogi Yarns – Yogi hits it out of the park on Wall Street

We continue our occasional series on open-source and “forking” with a hypothetical pitch by our master of the fork, Yogi Berra, in the form of a pitch to a venture capitalist with oodles of boodle hoping to make even more oodles…


Yogi: My son Yoda is a progammer. He’s very good at his job, and he and some friends have come upon a uniqure business opportunity, one that can make us oodles of boodle.

VC: Do tell! Tell me more.

Yogi: Yoda and his friends have spent years mastering a particular piece of internet technology. They are as good as it gets. This technology is used to power almost all the web servers on the planet. They are so good they have found a way to make it run twice as fast as the competition.

VC: Sounds exciting. Go on.

Yogi: They have written what is called a “patch.” If you apply this to the competitor’s software, then their version is twice as good, beyond dispute.

VC: But how do they get access to the competitor’s code? How much does it cost? How much do we have to pay?

Yogi: Nothing! Zippo! Nada! Or as I say, “”A nickel isn’t worth a dime today.” It’s free! It doesn’t get any better than that, does it?

VC: Unbelievable. How can this be possible.

Yogi: Those idiots did a good job, but they used some wacko technology called “open-source.” It’s freely available. I can download their code at no cost and apply Yoda’s patches. Those patches are like a light saber, cutting through the crappy code in their web server.

VC: I’m sold. How big a check do you need?

Yogi: I didn’t really say everything I said. But it’s still a good deal. How about ten million for seed money?

VC: It’s yours. Hit it out of the park.


Now go back and read that again, substituting “Bill Gates” for “Yogi.”

This is both the power and beauty of open-source.

Apache httpd server has been the dominant web server for almost a decade now.

For each and every day of that decade, Microsoft has had the option of forking the code, since Apache uses a very liberal license, yet they have never done so. Why not?

Let’s assume that the skilled programmers in Redmond could indeed come up with a patch that made httpd server run twice as fast in half the memory, a patch so good they could say to Apache, “Fork off, we’re outta here. You folks are toast.”

What would be their advantage?

None!

Even if they could write their patch, how long would it take the skilled programmers at Apache to figure out what they had done? Not long, these folks are good, very good. They would figure out what must have been changed, and would soon match the patch. I’m even confident they could write their way around any patent that Bill and friends could put together.

Never underestimate the skills of the best open-source programmers. They are, literally, as good as it gets.

This is why I am in a ways saddened whenever I see folks done the garb of Chicken Little and worry about the onslaught on commercial software as it goes toe-to-toe with open-source.

Bring it on! We can handle it! Not to worry.

The real reason folks use httpd server is that they know the folks behind it will be there a year from now, five years from now, or decades from now, standing behind their work each and every day.

It’s the community that counts, not the code.

Open-source is not about license, it’s not about code, it’s about the community that stands behind that code, and the opportunities that result once you engage with that community on its terms.

Yogi Yarns – Forks aren’t good or bad, they are just … forks

We continue our occasional series of open-source “forking” with an investigation of their morality, or lack of same.

I just came across an article arguing that a proposed fork is “bad.” See
Novell “Forking” OpenOffice.org
.

The post contains some strong language:

Novell is forking itself out of the FOSS community. Here’s the press release, to memorialize this day in FOSS history, and so you can reach your own conclusions.

I beg to disagree.

The ability to “fork” code is an option open to all. It is neither good nor bad .. it is just there. Time will tell whether the fork made sense or didn’t.

Whether to fork is a tactical decision, not a moral decision.

This is also the case in choosing a license. What matters is the community you hope to join or to create, and so you should elect a license that best meets what you perceive to be your needs.

Perhaps Novell is right. Perhaps they are wrong. Time will tell.

But who are we do deny them the right to take this path? The nature of open-source is that they have the option.

They have decided to use that option. What’s wrong with that? Nothing.

The folks who think it wrong are taking too literally one of the sayings of Yogi Berra:

“If you can’t imitate him, don’t copy him.”

All this is nothing new. We’ve been there and done that, over two hundred years ago.

If you walk down Fifth Avenue in NYC to its end, you will find yourself in the midst of the part of Manhattan known as Greenwich Village, in the midst of the campus of New York University.

You will also find yourself at the entrance to Washington Square Park, facing a large monument.

If you look up at the top of the monument, you will see words first uttered by George Washington:

Let us raise a standard to which the wise and honest can repair; the rest is in the hands of God.

President Washington uttered those words on the signing of the U.S. Constitution, but I think with a small change they apply equally well to open-source and forking:

Let us start a new project to which those in need of better code can repair; the rest is in our hands and the collective judgment of the open-source community.

Time will tell. The great advantage of open-source is that you can write your way to success .. or to failure. You are the master of your fate.

Yogi Yarns – Inbound and Outbound Licenses

As part of our going series on the notion of “forking” and open-source, let’s look into the notion of inbound and outbound licenses.

The “inbound” license is the license under which you which you receive an open-source package.

The “outbound” license is the license under which you make that code available, if you decide to redistribute it, in either modified or unmodified form.

Note that, in general, it is the act of distribution that triggers an open-source license. If you just download code that you receive under an open-source license and use it on your own, either for its intended purpose, or for the purpose of making changes and then using the modfied version, then you are under no further obligations.

While this concept is quite simple, I find it rarely mentioned in discussions of open-source. For example, I noted in an earlier post, Paul Courant, that one of the reasons I was impressed by a paper on open-source from the Ithaka.org group was that this notion was so clearly explained in a footnote. As it happens, I and one of the authors of that report took the same plane from Indianapolis back to New York after a conference in mid-October, and when I queried him about this, he said that this was common language among attorneys. (He was Barnaby Gibson, an attorney who works for ithaka.)

So one of the first questions to ask about the inbound license is, “What are my options in the outbound license?”

Some licences dictate some or all of the terms of the outbound license. For example, if the inbound license is GPL, then the outbound license must be GPL (see 2(b) in GPL License.) If the inbound license is LGPL, then the outbound license must be LGPL or GPL (see paragraph (3) in LGPL License).

It is the dictation by the inbound license of some of the terms of the outbound license that expresses what is informally known as a “viral” effect of a license, though in the case of these licenses there is also the additional consideration that the way in which your software is combined with the inbound software may dictate the terms under which the resultant work must be licensed.

So-called “viral” effects are not unique to GPL or LGPL. For example, both the Common Public License (IPL) and Eclipse Public License(EPL) have requirements that must be reflected in the outbound license.

Here, as always, I must add the disclaimer that I am not a lawyer, so you must consult with your own attorney for legal guidance. Also, I am speaking on my own, giving my personal views, and not those of my employer (IBM) in any way, shape, or form. On the other hand, the need for caution should not forbid our discussing these issues, as they are important to understanding open-source.

At the other extreme, so-called “liberal” licenses such as BSD and MIT say nothing about the terms of the outbound license other than that you must give a copyright notice as well as the terms of the inbound license.

Indeed, these licenses are “liberal” in a dual sense. You get to use the code as you deem fit, and you are also given freedom in writing the outbound license.

As a developer, I tend to prefer “liberal” licenses since I want to give the user as much freedom of action as possible. I don’t want to dictate how they must use the code, or how they must license it. I consider my work done when I release the code — what use to make of it is for others, not me, to decide.

This is also where I part ways with those who say that for code to be “open-source” it must be opened up on their terms, and not mine. I respect the licenses written by others, and I expect others to honor the terms of the license I decide to use.

You may not like my terms, but if you don’t, then go elsewhere in search of code, or write it your own code, on terms of your choice.

I don’t think I have the right to tell you how to license your code. Why should you have the right to tell me how to license mine?

Don’t bash me, or others who decide to use a liberal license — fire up a “bash” shell and start writing…

Yogi Yarns – On the definition of open-source fork

Wikipedia provides a reasonable definition of “fork” as it applies to open-source; see Fork (software development).

We will in future posts study some of the cases mentioned in the Wikipedia article, but first we should address a key deficiency in the Wikipedia article — it doesn’t explain how forks are possible.

We take as our definition of open-source the Open Source Definition (OSD), which begins as follows:

Open source doesn’t just mean access to the source code. The distribution terms of open-source software must comply with the following criteria:
1. Free Redistribution

The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale.

2. Source Code

The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost–preferably, downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed.

3. Derived Works

The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software.

(1) is minor. It just says that no money need change hands. Open-source must be freely available.

(2) says that for code to be open-source the source code must be available.

(3) is crucial. The phrase “must allow modifications and derived works” means that anyone can take open-source code, optionally make changes to it, and distribute the result, so long as they respect the terms of the license.

As a trivial point, you don’t have to make changes, so you can redistribute the code with no changes. But you can make changes if you are of a mind to.

It is (3) that means no one can claim ownership of a piece of open-source code. Any developer, including a developer working for a company, has as much right as anyone else to take the code and do with it as they will. You can take the gcc code from FSF, or the httpd server code from Apache, and take on the “big boys,” matching your skills with theirs.

It’s also worth noting that the notion of “fork” is central to open-source. As I have just shown, clauses (2) and (3) of the OSD enable forking. On the other hand, if you were to define “forking” as an essential attribute of what it means to be open-source, then you would wind up drafting language that would necessarily include clauses (2) and (3) in some form.

Simply, put “forking” is an essential part of the definition of “open-source,” and any definition of “open-source” must allow “forking.”

There are several kinds of forking, as well as some confusion about what they mean.

To fork an existing open-source project is to take its code and start another project based on the same code. The new project may just redistribute the same code, or it may add its own modifications, in the form of bug fixes, deletions, or by adding new functional enhancements, at the pleasure of the developers working on the new project.

By an “open fork” we mean that the new project decides to make the code as freely available as the old project. By a “closed fork” we mean that the new project takes another course.

The new project may, if the license allows it, take place within a firewall, by which I mean the project exists in a gated community to which the project maintainers control access. This is what most folks mean by close fork.

Some argue that a closed fork is inherently wrong, or evil, in that it denies the users of the new project some rights they had as users of the original project. These folks are in effect saying that all forks must be open.

We shall compare and contrast open and closed forks in future posts, but before doing so it is worth noting perhaps the most fundamental attribute of open-source, an attribute that is in some cases forgotten.

Open source is a form of writing, as is a poem, play, screenplay, or contract. As such, the author of the writing enjoys the copyright to that writing.

The author, the copyright owner, gets to decide the terms and conditions under which that writing may be made available to others.

(This is at least the case in the United States. Some other countries, notably Spain, say there are inherent rights in the use of writing not under the control of the author, but we will ignore those rights for this discussion.)

While some speak of “free software” as in “freedom” and others speak of “open-source software” as available under the terms of the OSD, there is one freedom that we must never forget.

The author has both the freedom to write and the freedom to decide the license terms of their writing. It is their call. To write is to create your own work, not to be bound by the beliefs of others. This is “freedom of the press” it its most essential form. To believe otherwise is to say that you have the right to dictate what others can write.

Thus, your decision about how to make use of the writing of another is simple, “use it or lose it.” If you agree to the terms of the license chosen by the author, then you are free to make use of their writing. Otherwise you cannot make use of their writing.

You may not like the license they have chosen, and you can try to convince them to release it under more liberal terms, but it is wrong to ignore their choice

But you do have to accept their right to make that choice. You either accept the terms or you don’t.

Yogi Yarns – On forking

This is the first of an occasional series of small stories, or yarns, on the concept of “fork” in the world of open-source.

Since this is such a complex and misunderstood topic, and in an attempt to keep your attention as we address this topic, we will seek inspiration from a number of sources, all beginning with the phrase, “Yo”.

Our chief guide will be Yogi Berra, the master of the clever phrase, as well as a Hall-of-Fame baseball player. and author of the classic work on forking, When You Come to a Fork in the Road, Take It!.

We will draw additional wisdom from Yoda, the wise master, strategist, and best duellist in history.

And to keep us going on this long diversion, we will take to the steps of the Philadelphia Museum of Art and consult a man almost as old in years as am I, Mr. “Yo” himself, Yo, Rocky, or Rambo, Gonna Fly Now at 60.

The story has many twists, left and right, up and down, and is a Yoyo of a story.

Musical accompaniment will be provided by Yoko Ono,not Lawrence Welk and his incessant Yodeling.

And of course we will set the table for this discussion by consulting Emily Post, mistress of etiquette, who, in a cautionary note, reminds us that

“Nothing is less important than which fork you use. Etiquette is the science of living. It embraces everything. It is ethics. It is honor.”

And since this is about programming, we will have occasion to delve into philosophy, and consult some of the world’s greatest philsophers, as we attempt our own solution of the Dining Philosphers Problem

Let us begin: Yo!

Paul Courant

I attended the conference “Licensing and Policy Summit for Software Sharing in Higher Education” in Indianapolis, Indiana, in mid-October. I have written of the conference in previous posts, and write this post to note my most pleasant meeting during the conference.

Prior to the conference a list of recommended materials was sent to the participants. One of the was a study from ithaka.org, which, in its own words,is a “not-for-profit organization in January 2004 with a mission to accelerate the productive uses of information technologies for the benefit of higher education around the world.”

One of their activities was the production of a study, OOSS (Organization for Open Source Software) Study, which produced a final report, Software and Collaboration in Higher Education: A Study of Open-Source Software. The authors of the report are Paul N. Courant, and Rebecca Griffiths. It includes contributions by Matthew Rascoff, Michael Carter, Kevin Guthrie and Barnaby Gibson.

I was quite impressed by the study. For example, the footnote on page 17 contains a discussion of “inbound” and “outbound” licenses. This is a notion I use often in my work, but don’t recall seeing so it clearly described in a report about open-source.

Prior to coming to IBM I spent just over twenty years at the Courant Institute of Mathematical Sciences (CIMS) at New York University (NYU), first as a graduate student and later as a research scientist. CIMS was the graduate school of Mathematics when I entered but later grew to include Computer Science as well.

CIMS takes its name from Richard Courant. He was a world-famous mathematician when he fled Nazi Germany in the early 1930’s. He founded what became CIMS and turned NYU into a major center of research in mathematics, much as Herman Wells, who also began his work about the same time, turned Indiana University into a major, international research univsersity.

So when I was introduced to Paul Courant at the conference, my first question was of course, “Are you related to Richard Courant?”

I was indeed delighted when he said that Richard Courant was his grandfather.

I only saw his grandfather a couple of times during my early years at Courant, as he was retired by then, but I did notice a definite resemblance between Paul and his grandfather.

Professor Courant has had a distinguished career, and you can learn more about him at his faculty page, Paul N. Courant.

Brad Wheeler kindly took a picture of the two of us, and I would like to share it with you (He’s on the left, I’m on the right):

October06 069

  • Pages

  • December 2006
    M T W T F S S
     123
    45678910
    11121314151617
    18192021222324
    25262728293031
  • RSS The Wayward Word Press

  • Recent Comments

    daveshields on SPITBOL for OSX is now av…
    Russ Urquhart on SPITBOL for OSX is now av…
    Sahana’s Respo… on A brief history of Sahana by S…
    Sahana’s Respo… on A brief history of Sahana by S…
    James Murray on On being the maintainer, sole…
  • Archives

  • Blog Stats

  • Top Posts

  • Top Rated

  • Recent Posts

  • Archives

  • Top Rated