Tag Archives: open-source

One Million Lines Per Minute: SPITBOL/386 times for tokenizing and code generation

I’m making good progress on producing a version of SPITBOL for Linux.

One sign I’m on the right track is the performance that PC-SPITBOL delivers when run on Linux using Wine.

The translation is done in two steps.

The first step tokenizes the input, puts the operations in a standard form and drops the comments. This takes about 0.8 seconds to scan 31,000+ lines of MINIMAL source for v3.7, and produces just over 12,000 lines of tokenized instructions. This corresponds to about 2,325,000 lines/minute.

The second step generates assembly code for 386 for the 12,000 instructions. This results in 14,000 386 instructions and takes about 1.4 seconds. The scales up to about 500,000 statements/minute for the code generation phase.

The combined phases take just over 2 seconds to translate 31,000 lines of MINIMAL to 14,000 386 instructions. This corresponds to about 930,000 source lines per minute.

Not bad for a program written over 25 years ago.

And that’s before doing any tuning…

SQL No, Set Theory Yes

Slashdot just posted a story about a “nosql” conference, Enthusiasts Convene To Say No To SQL, Hash Out New DB Breed.

The “nosql” folks seem to agree with me that SQL is fundamentally wrong.

However, a quick look at some of the presentations to be found at NOSQL debrief suggests they are missing a key point.

The presentations lament the problems with SQL in today’s world, and make mention of some of the latest buzzwords such as “JSON,” “REST” and “Log-structured Merge Trees.”

I suggest the answer is not be found in a mashup of hot new technologies but by going back to the basics.

That would be mathematics.

If set-theory is good enough for mathematicians shouldn’t it be good enough to describe what you want to retrieve from a database?

Get the basics right, and all the rest will follow. This is a much better approach than working backward from technology to determine how the user should construct a query.

PC-SPITBOL for Intel 8086 DOS Now Available

I just located my copy of PC-SPITBOL for the IBM PC, 8086, circa 1983. This is a 16-bit version.

It can be downloaded here.

Here are some pictures of the manual and a screenshot showing PC-SPITBOL starting up:

PC-SPITBOL Manual, 1983

PC-SPITBOL Manual, 1983

2009-06-NM-TREK 146

I did comparative runs of PC-SPITBOL 8086, SPITBOL/386, and CPython, using Cygwin on Windows XP:

CPython 55 seconds
SPITBOL/86 15 seconds
SPITBOL 8086 22 seconds

The numbers speak for themselves. PC-SPITBOL, written in 1983, is almost three times faster than CPython. SPITBOL/386, created over a decade ago, is more that four times faster than CPython.

Screenshot showing comparative times

Screenshot showing comparative times

Here are the test programs:

def add(a,b):
 return a+b
def sub(a,b):
 return a-b
n = 0
limit = 1000000
limit = limit * 50
trips = 0
for i in xrange(0, limit):
    m = add(n, 1)
    n = sub(m, 1)
    trips += 1

PC-SPITBOL, 8086, 16-bit:

 &stlimit = -1
 define("add(a,b)") :(add_end)
 add = a + b: (return)
 define("sub(a,b)") :(sub_end)
 sub = a - b: (return)
* repeat for limit1 * limit2 * limit3
 n = 0
* limit = 1000000
* limit = limit * 50
* variant for 16-bit spitbol, need to keep numbers small
 limit1 = 10000
 limit2 = 100
 limit3 = 50
 trip3 = 1
 trip2 = 1
 trip1 = 1
 m = add(n, 1)
 n = sub(m, 1)
 trip1 = trip1 + 1
 le(trip1, limit1) :s(lp1)
 trip2 = trip2 + 1
 le(trip2, limit2) :s(lp2)
 trip3 = trip3 + 1
 le(trip3,limit3)  :s(lp3)
 output = limit1 " " limit2 " " limit3

SPITBOL/386, 32-bit:

&stlimit = -1
 define("add(a,b)") :(add_end)
 add = a + b: (return)
 define("sub(a,b)") :(sub_end)
 sub = a - b: (return)
 n = 0
 limit = 1000000
 limit = limit * 50
 trips = 0
 m = add(n, 1)
 n = sub(m, 1)
 trips = trips + 1
 le(trips, limit) :s(lp)
 output = trips

There is a bug in the “sourcecode” function in that right parenthesis followed by a colon is deemed to be a “smiley face” and so rendered as an image. I find this so charming that I have left it in.

The St. Clair McKelway Test

In my previous post, , I posted some photos of the obituary of A. J. Llebling — the namesake of this blog — written by Tom Wolfe. Now a famed novelist and writer, Wolfe was then — in late 1963 — a young reporter for the New York Herald Tribune.

I just revisited this post, and on viewing the photos I posted of Wolfe’s work I noticed the name St. Clair McKelway!

Aside from A.J. himself, Mr McKelway was among my favorite writers from the New Yorker.

I suspect that not one in 10,000 or so working journalists would know his name.

I knew that Tom Wolfe was good, but I didn’t appreciate that Wolfe – or his editors at the Trib — were that good.

To include St. Clair in Liebling’s obituary was both an act of brilliance and a demonstration of profound understanding.

If you have ever read St. Clair’s work then please post a comment.

I’ll pay the bar tab if we ever meet, and I do hope we do so as soon as possible, so we can jointly pay tribute to Colonel Stingo.

Fun is fun, but this is serious business.

Good writers deserve recognition.

The Wayward Press: Tom Wolfe’s Obituary for A. J. Liebling in the New York Herald Tribune, December 1963

While rummaging through some old documents I came across a copy of the obituary for A. J. Liebling written by Tom Wolfe. Now a famed novelist and writer, Wolfe was then a young reporter for the New York Herald Tribune. This copy was the gracious gift of Harold Rosenberg, a legendary sportswriter in New York in the 40’s and 50’s. I met him at a family gathering of my college roommate, Robert “Buzz” Bernstein, back in the 60’s. I had mentioned my interest in A.J.to him, and Mr. Rosenberg said that Wolfe had written A.J’s obituary for the Trib. He then volunteered to dig up a copy from the Trib’s “morgue” , which I received a few weeks later. [1]

Mr. Rosenberg was — as is almost always the case with experienced journalists — a wonderful man. He later was the chief publicist for the New York Jets in their early days. I regret that I met him only this one time.

Journalists are among the least-appreciated craftsmen in our society. They dedicate their lives to learning how to tell stories in simple prose. We take their artistry for granted.

As a professional programmer I regret deeply that the writing of code is now deemed more valuable than the writing of journalists.

Reporters, like skilled programmers, are a dwindling breed.

I fear we shall appreciate the important role of journalists only when their craft has become a lost art, as the internet-driven process underway that is destroying print journalism nears its end.

The assault of technology on journalism is not a new story. For example, one of my closest friends from my high school days in Albuquerque, New Mexio, was Gene Griffith. His father, Paul Griffith, was both a newspaper publisher and one of the nicest people I have ever met. Born in Iowa, Paul was a long distance runner, as were two of his sons, including Gene. He moved to New Mexico, and was the publisher of a newspaper in Eastern New Mexico when Buddy Holly was killed in a plane crash.

When I knew him, Paul Griffith spent his days being an outstanding person and also the operator of a linotype machine at the Albuquerque Tribune. It might have been the Albuquerque Journal, but that doesn’t matter, since one of the landmark decisions of the Supreme Court in the 1930’s was to allow the Journal and Tribune to merge their operations, so that both papers could survive.

My single most memorable memory of the craft that was required to publish a newspaper came one evening when Gene took me on a visit to see his Dad at work.

I have seen many wondrous technologies in my life, but the two that stand out are these:

  1. Seeing a black-and-white photograph develop under one’s eyes in a darkroom, as a chemical process that reaches down to the molecular level unfolds its magic;
  2. Seeing an ingot of lead fed into a linotype machine and watching hot type emerge at the other end. [2]

It was while watching Paul at work that I came to appreciate the magic that was print journalism, back when A. J. Liebling was one of its craftsmen. The former I learned a few years later when I had my own darkroom, and I think it worth noting that no digital technology can ever match the glory that comes from large glass plates using traditional photographic processes.

Computers are nice, but silver halide molecules are even nicer!

There is nothing, save perhaps the printing press itself, that comes closer to technological perfection than seeing cold metal turned into hot type.

Setting technology apart, I agree with Red Smith that the closest man has come to perfection is “90 feet from home to first base.”

As a example of a good reporter at work, here are some photos of Wolfe’s obituary for Liebling:

For example, note Wolfe’s mention of Ibn Khaldun, a famed Arabian historian, and author of the “Muqadimmah.” I once mentioned Liebling’s fondness for this work to Jack Schwartz. Jack said a few weeks later that he had read the work, and agreed that it was quite good. (I have read only part of it, and need to revisit it when time permits.)

2009-06-NM-TREK 137

2009-06-NM-TREK 138

2009-06-NM-TREK 139


1. “Morgue” is the set of old clippings of articles published by a newspaper, “back in the day.”

2. Samuel Clemens, “Mark Twain,” was so seduced by this technology that he almost bankrupted his family, and so had to take to the road to give lectures. I vividly recall Hal Holbrooke’s solo performance of “Mark Twain Tonight” that I saw around 1960, when I was fifteen or so. You can distinguish those who have seen that performance by asking about “Halley’s Comet.”

Perfect Attendance Junkies

I am one of a select group of people who went one or more years without missing a single day of school. For example,

My fourth PA (Perfect Attendance) Certificate.

My fourth PA (Perfect Attendance) Certificate.

I didn’t miss a day of school from the start of third grade until at least the second year of my graduate education, a run of sixteen — and perhaps more — years.

Please post a comment if you can make a similar claim.

WordPress Continues to Amaze: May 2009

Here is an excerpt from the WordPress May Wrap-up:

WordPress rolled out a ton of new features: the ability to add YouTube videos and polls to comments, stats in your time zone, the VideoPress upgrade (with HD!), post by email, new stats charts, comment search, improved comment reply by email, and VideoPress.com.

The stats for May:

* 387,416 blogs were created.
* 411,704 new users joined.
* 5,504,742 file uploads.
* 3,333 gigabytes of new files.
* 826 terabytes of content transferred from our datacenters.
* 6,914,546 logins.
* 1,243,177,638 pageviews on WordPress.com, and another 1,207,143,849 on self-hosted blogs (2,450,321,487 total across all WordPress blogs we track).
* 2,105,723 active blogs where “active” means they got a human visitor.
* 1,728,890,160 words.

Let’s divide by 30 to get daily figures:

* 13,000 blogs were created, EACH DAY.
* 13,700 new users joined, EACH DAY.
* 183,000 file uploads, EACH DAY.
* 111 gigabytes of new files, EACH DAY.
* 27.5 terabytes of content transferred from our datacenters, EACH DAY.
* 286,000 comments, EACH DAY.
* 230,000 login, EACH DAY.
* 41.4 million pageviews on WordPress.com, and another 40.2 million on self-hosted blogs (81.7 million total across all WordPress blogs we track, EACH DAY.
* 57.6 million words, EACH DAY.

Let’s divide that by 1440 (24*60) to get the stats for EACH MINUTE:

* 9.2 blogs were created, EACH MINUTE.
* 9.5 new users joined, EACH MINUTE.
* 127 file uploads, EACH MINUTE.
* 77 megabytes of new files, EACH MINUTE.
* 19.1 gigaabytes of content transferred from our datacenters, EACH MINUTE.
* 198 comments, EACH MINUTE.
* 160 logins, EACH MINUTE.
* 29,000 million pageviews on WordPress.com, and another 3,000 million on self-hosted blogs (56,700 total across all WordPress blogs we track, EACHMINUTE.
* 4,000 words, EACH MINUTE.

Keep up the good work, WordPress Team.

And, of course, thanks to my fellow WordPress bloggers. It is the content that we generate, each and every minute, that keeps WordPress growing, both in size and in value.

This is yet another reminder that WordPress is by far the best blogging platform, and also a reminder that anyone using another blogging platform comments on their lack of savvy with each and every blog post.

This is also a case where bigger is better. For example, imagine the predicament of the ruling powers in Iran. My guess is that they cannot possibly selectively filter all the WP posts about them, so the will have to keep it all going, or shut it all down.

We’ll know their jig is up when they resort to the latter course of action.

This is also a reminder that the best programs — and the programmers who write them — are to be found in the open-source arena. That’s where the big boys play.

We do it for the fun of it, which is why we give it away.



100 Compiler Options and You are Out

California, along with Major League Baseball, is among the institutions that have a policy of “Three Strikes and You are Out.”

I suggest the same rule should apply to compiler writers.

For example, I just tried to port some software that makes use of gcc, the standard Linux C compiler, only to get an error message about a problem with an option.

I then did “man gcc” to review the list of options. As best as I can tell, there are well over a hundred of them.

This is ridiculous. How can any person be expected to make meaningful use of such a superfluity of programming madness.

Let’s do the math. I once read there are about 2**80 atoms in the universe. If there are 100 options to gcc, then are there 2**100 ways of compiling a program, which is 10**20 more times options than there are atoms in the universe!.

If we assign one Richard Stallman to test each possibility, then we need at least 10**20 RMS’s just to test the path flow for options!

GCC Folks: Get your act together. If you can’t come up with a small, reasonable set of options, then I suggest you should leave the compiler game and let the grownups take over.

Four colors suffice to color a map. I’ll give you double that, so can you please cut the number of options down to 8?

  • Pages

  • July 2022
    M T W T F S S
  • RSS The Wayward Word Press

  • Recent Comments

    daveshields on SPITBOL for OSX is now av…
    Russ Urquhart on SPITBOL for OSX is now av…
    Sahana’s Respo… on A brief history of Sahana by S…
    Sahana’s Respo… on A brief history of Sahana by S…
    James Murray on On being the maintainer, sole…
  • Archives

  • Blog Stats

  • Top Posts

  • Top Rated

  • Recent Posts

  • Archives

  • Top Rated