robjsoftware.info

Sorry for the logorrheic posting flurry. I’ve been wanting to start a blog for years but for some reason never did. Now that it’s up, the dam has burst.

So this post is about my history as a programmer, and about some of the lessons I’ve learned in ambition and tackling the right problem.

First let me say that I’m hopelessly in love with programming. My basic definition of software engineering is “building machines out of ideas.” And that’s something that essentially has no bottom, and no top. There’s no limit to software.

That flexibility is also a trap, because in reality software has to be built on other software. And overreach is the ambitious engineer’s curse.

Let me give some examples from my own career. My first real post-college job, in 1990, was working on Project Xanadu. I’d read Engines of Creation in school, and it totally changed my life. I wanted to make hypertext happen, now.

So I flew out to California and started working for Autodesk with the Xanadu folks. What followed was one of the most exhilarating, and then one of the most disillusioning, periods of my life.

Xanadu was a glorious triumph of software design, and a consummate failure of software production. Picture it: the entire system was predicated on the assumption that hyperlinks would only work if they were fundamentally coupled to the individual textual characters that made up their anchor, such that any editing operations would exactly track which characters were linked to. The design sought to create a serverless, versioning system for generic hyperdimensional multimedia data, on top of which was an addressing scheme for arbitrary hyperlinks. All this was built on a super secret data structure, essentially a multi-coordinate splay tree (with adaptive optimization for all lookups). An object-oriented data store provided the persistence. A pipelining message protocol provided the networking And all of this was written in Smalltalk, compiled to C++ for efficiency.

IN 1991.

Do you remember how slow computers were back then?

The World Wide Web came along a couple years after the project finally fell apart, and took over everything. Because the fundamental assumption of Xanadu — that the critical problem was tracking all anchors across all content edits — was wrong. That problem wasn’t critical. And Xanadu’s now a footnote.

On the bright side, I have a lot of friends from that era, despite my hastily chosen and immature words in the Wired article that came out in 1993. Dean Tribble and Ravi Pandya now live in Seattle, working on the Singularity OS research project at Microsoft. Mark Miller is at HP now, thinking more deeply about secure mobile code than anyone. Life goes on and ideas flower in their time.

So from that, I decided to reduce the ambition level a lot, and went back to Autodesk to work on Autodesk Animator Studio, a Windows video paint package.

For Windows 3.1.

Anyone remember what watching video in Windows 3.1 was like? On a 1994 Pentium with 8 megs of RAM? Now imagine editing it.

(It looks like the only little video I saved from those days no longer plays in Windows XP. Sigh, bit rot creepeth ever unto the past.)

Yes, this could also be called overly ambitious. We did ship, but the whole experience drove my manager at the time, Jim Kent, out of the software biz for good. Fortunately, six years later, that turned out to be a very good thing for the public version of the Human Genome Project.

But hey, shipping was a big step forward!

A brief detour into charting software, and then I decided I wanted to get into the computer game business. So I worked for a small company named Protozoa in San Francisco. Protozoa was doing performance animation software, and wanted to make a game with their technology. I learned a lot about 3D and realtime programming there. I also learned that if you don’t have a total focus on what makes your game fun, your game will not in fact be fun. I ALSO learned that building a realtime 3D game on DirectX 3, in 1995, is not the best idea. Again, ambition bites, hard.

Years later I learned that one guy I worked with there, Bay Raitt, was destined for glory. Bay was a super genius character modeler. He was agonized by the problems of trying to build characters with only 200 polygons. Exactly how agonized I only realized years later, after seeing the second Lord of the Rings movie. I bought a book about how they created Gollum, and lo and behold, Bay was Gollum’s lead facial animator. In retrospect, seeing him work on a 1995 3D computer game was like seeing Picasso trying to work with a half-dry jar of finger paint.

Alrighty then! Time to move on from computer games, I thought. But the stings of Xanadu had healed somewhat, and I wanted to learn about networking. And a number of the Xanadu folks had re-convened in Cupertino, at a startup named Electric Communities. EC was seeking to take some Xanadu-era ideas about capability security and secure mobile code, and to build from them a distributed, peer-to-peer, cryptographically authenticated, secure-identity-supporting, commerce-capable virtual world.

Using Java version 1.0.

In 1996.

Are you seeing a pattern here?

EC failed for reasons similar to Xanadu: the software was too big, too ambitious, and too slow to make a compelling product. Not only that, but users didn’t want virtual worlds nearly as much as everyone thought. Years later, and with 21st-century hardware, only Second Life has somewhat cracked the nut, and even their infrastructure is suffering pretty deeply right now (word has it).

Again, though, I met a number of super smart people who are now doing great things elsewhere, including Doug Crockford, moderately (ahem) well-known in Web 2.0 circles. Also Arturo Bejar, security ninja at Yahoo; Brian Marick, a major testing god; and Chip Morningstar and Randy Farmer, also now at Yahoo, who have a quite candid assessment of the whole era on THEIR blog. And I got to spend the better part of a year figuring out how to create a secure, capability-based protocol for virtual-world object transfer between untrusting hosts. Getting paid for that? It was really, really fun. Until the money ran out.

EC was really the end of my over-ambitious days. In 1998, after EC, I went on to work for Helium, a now-defunct consulting company founded by an old Protozoa buddy (Gever Tulley) and a number of his oldest friends. We did a bunch of work for Quokka.com (a truly classic dot-com flameout story — they picked the wrong sports to cover and created all their sites by hand, but we had fun doing Internet livecasting of sailing races and the 2000 Olympics!). Then Quokka folded in 2001, Helium reformed, and we went on like that until 2003.

Finally in 2003, I begain working for my current employer, Nimblefish. A number of my old Helium and Quokka friends joined me there, along with other great folks. I’ve been there for three and a half years now, an all-time record for my career, and we’ve got a working production system that just keeps getting better and better. Hibernate, Flex, and other modern tech drives our code; we’ve got real customers, and we’re growing; and we’ve got a clear strategy that we’re executing on at full bandwidth. I think I might have finally kicked the curse of over-ambition…..

…or maybe it’s just dormant! 🙂

I’ll close with the word that best captures how I feel about programming:

Onwards!