robjsoftware.info

It’s almost terrifying how busy things are right now in robjsoftwareland. Moving with two young kids is just not the best idea from any kind of sane logistical perspective. Trust me on this. PLEASE. My life is a continual rolling boil of domestic packing and repacking and remodeling and general chaos, and it will continue for months to come.

But I cannot and will not go dark on this blog. So, a quick update:

1) The video from GWT2007 got posted! If my previous posts about my RPC talk made you curious, you can now see for yourself. I’m amused at the Sartre reference that snuck into the video 🙂 I think it came out quite well overall (with the caveat, as before, that an unexpectedly large portion of the audience were newbies who didn’t get the most out of it). The other talks are also online — now I’ll get to see the ones I missed! I’d also recommend Billy Hoffman’s talk on security (warning: one short NSFW item in there!).

2) I’m currently putting all research papers and language-y thinking on hold for a few months. Instead I’m getting back to basics. All my side reading/hacking time is going into plowing through these two books: Introduction to Algorithms and Artificial Intelligence: A Modern Approach. I’ve generally in the past spent most of my time reading current research, because books get out of date so quickly. Well, not these two! They come pretty darn close to being timeless.

One quick brain teaser from the algorithms book: Everyone knows how to write a recursive algorithm to print an inorder traversal of a binary tree. Almost everyone knows how to convert it to an iterative algorithm using an explicit stack, too. But the book mentions that — if your binary tree contains parent links as well as child links, and if you can test pointers for equality — you can actually write an iterative algorithm to do an inorder traversal without keeping an explicit stack… in other words, an iterative algorithm that uses constant space (rather than space proportional to the depth of the tree). It took me about twenty minutes to figure it out. How about you?

3) One of the more interesting posts recently on good old Lambda the Ultimate was this post by Paul Snively asking whether syntactic or semantic theory is the way of the future. Most approaches to programming language theory historically have been largely syntactic, but there are a number of recent papers that take a purely semantic view.

While I’m still learning this field myself (and haven’t spent nearly enough time on the basics — Pierce’s syntactically-oriented Types and Programming Languages is on my reading list, but so far I’ve only dabbled), it seems to me that semantic approaches have an intentional stance that might make them fundamentally more general. For example, this paper on semantic verification of a simple compiler makes the point that a typed assembly language program can only go wrong if it tries to treat a given memory location as being of an unsafe type, but that there might be possible values in that memory location which can be safely interpreted as other types without inconsistency.

This approach reminds me of the Logic of File Systems paper, in which file systems are modeled in terms of their beliefs about the contents of memory versus disk. It’s a fascinating philosophical view, to think of programs as enacted, operational beliefs and to consistency check them on that basis. A program only goes wrong if its beliefs are inconsistent with the contents of the machine, as opposed to being inconsistent with its own source text (represented as base terms of the program). It’s going to be fascinating to see how far the semantic approach can be pushed.

That’s it for early January — see you in about two weeks!

Sigh, another two weeks gone by without a post. Still, that last post was a doozy, so cut me some slack, willya? Incidentally, my readership has increased by 50% in the last two weeks according to Feedburner. I am honored to have all of you paying attention to me, and I will try to be worthy of your eyeballs and neurons.

This post is going to be a bit of a catch-all: some about me, some about my last big-ass posting, and some about the GWT conference.

First, sorry I missed last week — I got another cold (endemic in the wintertime with two young kids in the house), and my wife kicked into moving high gear. Yes, we’re selling our house, for a wide variety of reasons I will discuss in personal email if you’re interested enough to send me some. (Not in the comments, sorry!) We’re shooting for having it on the market March 1, which means there is no time at all to waste. I’ll be doing no hacking for the next two and a half months, that’s an absolute given. I’ll keep up the blogging, but weekly may be tough; bimonthly WILL happen if it kills me!

Which is all too bad, in a way, because there’s plenty to do in hackland. My manifesto was rather well received. The comments on the post itself have a number of intriguing links. I also posted about it on Lambda the Ultimate, where it got a number of other fascinating responses. Thanks to all who engaged with it in either location. I am sorry I won’t be able to touch it until March first at the earliest, but that’s the nice thing about research — interesting ideas don’t go away!

I also wanted to give an update on the GWT conference. My talk was quite well received if the consensus speaker’s feedback was anything to go by. I appreciate the appreciation, to say the least. I did hear that there was a significant fraction (over 50%?) of the audience that felt it was too technical. Evidently a substantial quorum of GWT2007 attendees were relatively unfamiliar with GWT at any level. I apologize to anyone who wasn’t experienced enough to get the most out of my talk; I hope that you can find helpful material on the GWT site and/or any of the various GWT books. I’ve put my slides online for anyone interested. Evidently it’ll be Youtubed sooner or later, but I’ve yet to get a date on that.

The last morning of the conference, we had a participatory breakfast session where Joel Webber and I led a discussion of declarative UI and declarative data binding in GWT. One thing that was clear from my talk is that there are a lot of people doing form-like web applications, and GWT underserves that population right now. Joel spent some time going over the declarative UI work he’s doing; he specifically mentioned that various internal Google customers want him to ensure that you can write a GWT-XML document that lays out a GWT widget tree with HTML fragments mixed in almost arbitrarily. This seems like a win insofar as Joel’s talk on performance specifically mentioned that innerHTML is your friend; the DOM is faster at inserting HTML fragments than it is at handling individual DOM element insertions. So everyone keep an eye on Joel’s stuff as it hits the incubator.

I asked the table what kinds of form-data-binding approaches would be most useful to implement. The consensus seemed to be split. A number of people were familiar with the JSF-like structure of binding to backing beans and using expression language to specify the coupling from bean fields to the UI. I mentioned JSR 295 and various people expressed interst in that. There is a recent thread on the GWT contributors forum about some community work in this very area, which I look forward to investigating in depth (in early March! 😛 ). Check that thread out if you want GWT to do this. I talk at more length there about a potential GWT EL parser.

There was one gentleman (whose card I got, and subsequently lost; please contact me again, sir!) who was a forceful advocate of an XForms-like approach; evidently XForms lets you declaratively specify data dependencies within your model, and allows you to lay out interactions witihn your UI purely in your interface specification. This was pretty much news to everyone at the table. I will definitely have to look into XForms now. However, it seems to me that a JSF-like, JSR-295-like bean-binding approach covers many of the basic use cases people want, and should certainly be a near-term project.

OK, that’s it for this week. Happy, happy holidays to you all!

I’ve been working on a truly monster blog post, nothing short of a manifesto for a growable programming language. It’s about four pages long now and roughly 90% complete. All I have to do is Google up the two dozen reference links.

But I can’t quite get to it tonight, because tomorrow is my presentation at the GWT conference and I’ve been tweaking my slides semi-compulsively.

So, hopefully I’ll see several of you tomorrow at the conference, and stay tuned later this week for my Big Fat Language Rant! Your patience is very deeply appreciated.

(Warning: relatively low technical content ahead!)

Rats, missed another blog-posting week here. A week ago I finally got my GWT conference presentation done. I was rather nervous about whether I was hitting the right technical level, so I mailed it to a few GWT team members, and Bruce Johnson (the team lead) wrote back and said, basically, “It’s perfect.” Thanks, Bruce! Eased my mind tremendously. The conference is coming up next week.

Then on Tuesday, Mass Effect shipped, and I was instantly lost in space. I’ve gone back and forth on whether to talk about gaming here, but this title pushed me over the edge. I’ve been a computer gamer ever since I’ve been a computer user — my very first experience with a computer was playing Spacewar on a PLATO timesharing terminal at my friend’s father’s college. And gaming has become a true lifelong hobby. Mass Effect is a great example of why that is.

Programmers inherently love systems. A truly intrinsically motivated programmer will spend hours thinking about how something fits together, and more hours tinkering with it and seeing what happens. A great game also has a systemic internal structure that rewards tinkering and exploration. So it’s natural that so many programmers are gamers.

Moreover, many programmers are thrilled by advances in technology. The more powerful our computers become, the more we can do with them. And computer games are definitely the most widely visible hardware-pushing applications. Just take a look at some of these screenshots. A $280 XBox 360 can now do real-time facial animation that looks a lot better than ANYTHING being done five years ago (real-time or pre-rendered). And it qualitatively changes the nature of the game, when your character and the zany aliens they’re talking to both have facial emotions and lip-syncing that are so much closer to realistic that you can almost lose yourself in the illusion. I’m not claiming that Bioware’s crossed the uncanny valley just yet, but they’ve definitely taken huge strides in that direction, and they’ve hooked me but good.

I expect to be a gamer for the rest of my life, since games just keep getting cooler and cooler as hardware gets more and more powerful and we learn more about what to do with it. It’s a perfect hobby for a systems-oriented, game-addicted technophile.

So Mass Effect took over my after-the-family’s-asleep life, and stole my blogging cycles last week. Thanksgiving was also pretty hectic — we cooked an 18-pound turkey for our church get-together, which took me all day while my wife juggled our three-month-old and our (almost-)three-year-old. That left us pretty wrecked and we laid low the rest of the weekend.

Meanwhile, on the hacking front, I have continued to feel blocked by the whole Seam 2 broke my code issue. Actually, technically, it’s that Seam 2 broke the G4JSF code. That code was originally written by the Ajax4JSF team, but they have abandoned it. So the question for me is, how motivated am I to fix it? (And, potentially, to continue fixing it as Seam, JSF, and GWT continue to evolve?)

I originally started on this whole Seam / GWT / JSF project because I had ambitions to create a peer-to-peer distributed blogging system. I made a lot of progress on that system (at least as far as creating a Seam-based model for representing version trees with Java persistence, a la Mercurial). Then I wanted to start on the UI. Which led me to GWT, and the realization that the existing G4JSF library didn’t couple gracefully with GWT RPC. So I patched GWT, and then G4JSF, and that led me to presenting to the GWT team at JavaOne last year, which then got me the opportunity to speak at the GWT conference next week. All of that was quite unexpected and quite appreciated — particularly the speaking opportunity!

BUT, going back to my original project, the peer-to-peer distributed object system lost its mojo somewhere over the last year. It led me down the GWT / Seam / JSF path, which was really interesting and connected me to the open source world like never before… but somehow the journey became the destination, and the original goal no longer feels as interesting. Frankly, I’m not feeling very enthusiastic about being the sole owner of the G4JSF code base, given the evident cost of keeping it current and given the lack of other support from the JSF community. Maybe having a second baby also cut my energy level.

In order to avoid burnout I need to work on the projects that are most interesting to me, which right now is looking like prototyping some extensible programming language ideas that have been burning a hole in my brain for the last two months.

So, I’m going to set aside the G4JSF work and I’m going to leave the Seam/GWT/JSF integration project in its current state. I am more than willing to work with anyone (at Red Hat or elsewhere) who wants to pick it up again and fix it up to work with Seam 2 (and JSF 2 when it comes along), but I can’t drive it further on my own. Anyone who was using that integration library and who is interested in fixing it, feel free to email me (rjellinghaus at gmail dot com).

This blog will shift focus away from Seam/JSF and towards programming language research and the experiments I’m doing. Hopefully that’ll still be interesting to my loyal readers — I’ll do my damnedest to make it so! Because I’ll tell you one thing, it’s pretty fascinating to me 🙂

Unless you’ve been buried under a boulder, you’ve heard about Android. It’s nice to know what my secretive Google mobile JVM guru friend has been working on all this time 🙂

The Android forum is awash in C / C++ developers ranting about how lame Android is because it doesn’t support raw access to the hardware. It’s too bad, really. I feel for them, and I agree: Android apps are going to be slow and clunky compared to native apps.

But Google’s making a play to build a whole new kind of mobile ecosystem here. Google’s betting on increasing mobile CPU power (certainly an excellent bet), and Google knows that if Android develops a personal-computer-like reputation for flakiness, it will fail and fail bigtime. I would argue that a big reason no mobile platform has really taken off yet (besides crazy licensing costs) is that native apps are simply flaky, and the more of them you run, the flakier your hardware gets. Since Android is trying to greatly increase the hardware diversity of Android devices, allowing raw native access combined with open developer access would rapidly drive the stability of the system to zero.

Native developers hate to hear this, because they spend their lives tuning for performance and chasing crash bugs. I’m sure many of them do write good solid apps. But I’m also sure that making native code crash-proof, especially across a wide spectrum of hardware, is very, very hard to do. And if you want to throw your platform open to anyone who wants to ship an app for it, you’re guaranteed that many if not most of those people will lack the uber native skillz to achieve that kind of cross-platform stability.

So HOORAY GOOGLE! for making Android a fully managed platform! Note also that Microsoft sees the writing on the wall; they’re working on a fully managed operating system, for crying out loud.

End rant. The most interesting thing I’ve yet learned about Android is that it is fundamentally trying to end-run around Sun’s IP restrictions on Java ME. Stefano Mazzocchi breaks it down in exemplary fashion. Super cool! It’s going to be really strange to see Sun and Google at each other’s throats, but given how close the two companies are, and given how heavily Android leverages Java-the-language while totally subverting Java-the-runtime, AND given how anti-patent-war Sun is trying to be, I hope for some kind of peaceful reconciliation there. Microsoft, on the other hand, will go apeshit. Let the fireworks begin!

[Crossposted to both my blogs]

I’m overjoyed to announce that at 2:05 AM on Monday, July 30, our son Matthew Thomas was born.

He’s a 9 pound 5 ounce bundle of joy. He’s sleeping well, nursing a lot (gaining his ounce a day — hardly lost any weight after birth), and his big sister loves him, as you can see 🙂

Michelle, my wife, is resting and recovering, along with taking wonderful care of him. I’m taking time off work to care for them both, as well as for our daughter Sophie.

We are incredibly blessed. And what’s more, our family is now complete. After having two oversized kids, with difficult births both times, we’re done… we’re going to count our blessings, quit while we’re ahead, and move from child-birthing to child-rearing for good.

PHEW!!!!!

Thanks to everyone who’s sent congratulations and other good wishes.

What’s more, we’re even getting down to enough of a routine that I can think about blogging and hacking again 🙂 It’d go a bit more quickly if I hadn’t just gotten a sore throat… but anyway, stay tuned.

My wife is within days of delivering our second child. We’ve already got a two-and-a-half year old girl, Sophie, who’s a little bundle of shining happiness — seriously, this kid is one of the cheeriest tots you’re likely to meet, and my wife and I are totally smitten with her. Well, now her baby brother Matthew is on the way, and it’s happening Real Soon Now.

So at some point in the not-very-distant-at-all future, I’m going to be going offline in a big way, possibly for up to a month. Do not fret. I shall return in force, once we’ve gotten the hang of juggling TWO little ones in the house, and once we’ve made darn sure that Sophie feels OK with everything.

Many people love Perforce. That includes me.

What a lot of people don’t realize, though, is that if you are an individual programmer working on your own personal projects, Perforce is free.

See, you can download the latest release of Perforce and use it in an evaluation mode, where it’s limited to two users and two clients.

If you’re only using it to maintain your own local changes to one or more open source projects (say, GWT and Ajax4JSF and Seam), then that’s all you need. There’s no limit on the number of files or branches, and all Perforce features are 100% functional.

Right now on my Windows PC, I’ve got a Perforce depot which has:

– a complete download of the GWT Subversion tree
– a complete branch of that GWT tree, with my changes in it
– a complete download of the Ajax4JSF Subversion tree
– a complete branch of THAT Ajax4JSF tree, with my changes in it
– a copy of the Seam booking example
– a branch of the Seam booking example, with my changes in it

Setting my Windows Perforce client to use Unix line endings means that Perforce doesn’t screw up the Subversion metadata.

The workflow is:

– check out from Subversion
– check in to Perforce
– branch in Perforce
– edit like mad in the branch, committing at will
– once done, integrate back to the Subversion copy
– check out the whole Subversion copy for edit
– svn update, svn patch, svn commit
– revert unchanged files in Perforce
– re-commit to Perforce

The truly wonderful part is that I can take changes from the Subversion copy to my dev branch, or vice versa, as circumstances warrant. Basically, it’s the full power of Perforce branching, used to manage my personal development alongside the concurrent development of everyone who’s landing in Subversion.

I can’t recommend it too highly. Totally brain-saving compared to keeping multiple local Subversion working copies or whatever. Try it, you might love it.

Sorry for the logorrheic posting flurry. I’ve been wanting to start a blog for years but for some reason never did. Now that it’s up, the dam has burst.

So this post is about my history as a programmer, and about some of the lessons I’ve learned in ambition and tackling the right problem.

First let me say that I’m hopelessly in love with programming. My basic definition of software engineering is “building machines out of ideas.” And that’s something that essentially has no bottom, and no top. There’s no limit to software.

That flexibility is also a trap, because in reality software has to be built on other software. And overreach is the ambitious engineer’s curse.

Let me give some examples from my own career. My first real post-college job, in 1990, was working on Project Xanadu. I’d read Engines of Creation in school, and it totally changed my life. I wanted to make hypertext happen, now.

So I flew out to California and started working for Autodesk with the Xanadu folks. What followed was one of the most exhilarating, and then one of the most disillusioning, periods of my life.

Xanadu was a glorious triumph of software design, and a consummate failure of software production. Picture it: the entire system was predicated on the assumption that hyperlinks would only work if they were fundamentally coupled to the individual textual characters that made up their anchor, such that any editing operations would exactly track which characters were linked to. The design sought to create a serverless, versioning system for generic hyperdimensional multimedia data, on top of which was an addressing scheme for arbitrary hyperlinks. All this was built on a super secret data structure, essentially a multi-coordinate splay tree (with adaptive optimization for all lookups). An object-oriented data store provided the persistence. A pipelining message protocol provided the networking And all of this was written in Smalltalk, compiled to C++ for efficiency.

IN 1991.

Do you remember how slow computers were back then?

The World Wide Web came along a couple years after the project finally fell apart, and took over everything. Because the fundamental assumption of Xanadu — that the critical problem was tracking all anchors across all content edits — was wrong. That problem wasn’t critical. And Xanadu’s now a footnote.

On the bright side, I have a lot of friends from that era, despite my hastily chosen and immature words in the Wired article that came out in 1993. Dean Tribble and Ravi Pandya now live in Seattle, working on the Singularity OS research project at Microsoft. Mark Miller is at HP now, thinking more deeply about secure mobile code than anyone. Life goes on and ideas flower in their time.

So from that, I decided to reduce the ambition level a lot, and went back to Autodesk to work on Autodesk Animator Studio, a Windows video paint package.

For Windows 3.1.

Anyone remember what watching video in Windows 3.1 was like? On a 1994 Pentium with 8 megs of RAM? Now imagine editing it.

(It looks like the only little video I saved from those days no longer plays in Windows XP. Sigh, bit rot creepeth ever unto the past.)

Yes, this could also be called overly ambitious. We did ship, but the whole experience drove my manager at the time, Jim Kent, out of the software biz for good. Fortunately, six years later, that turned out to be a very good thing for the public version of the Human Genome Project.

But hey, shipping was a big step forward!

A brief detour into charting software, and then I decided I wanted to get into the computer game business. So I worked for a small company named Protozoa in San Francisco. Protozoa was doing performance animation software, and wanted to make a game with their technology. I learned a lot about 3D and realtime programming there. I also learned that if you don’t have a total focus on what makes your game fun, your game will not in fact be fun. I ALSO learned that building a realtime 3D game on DirectX 3, in 1995, is not the best idea. Again, ambition bites, hard.

Years later I learned that one guy I worked with there, Bay Raitt, was destined for glory. Bay was a super genius character modeler. He was agonized by the problems of trying to build characters with only 200 polygons. Exactly how agonized I only realized years later, after seeing the second Lord of the Rings movie. I bought a book about how they created Gollum, and lo and behold, Bay was Gollum’s lead facial animator. In retrospect, seeing him work on a 1995 3D computer game was like seeing Picasso trying to work with a half-dry jar of finger paint.

Alrighty then! Time to move on from computer games, I thought. But the stings of Xanadu had healed somewhat, and I wanted to learn about networking. And a number of the Xanadu folks had re-convened in Cupertino, at a startup named Electric Communities. EC was seeking to take some Xanadu-era ideas about capability security and secure mobile code, and to build from them a distributed, peer-to-peer, cryptographically authenticated, secure-identity-supporting, commerce-capable virtual world.

Using Java version 1.0.

In 1996.

Are you seeing a pattern here?

EC failed for reasons similar to Xanadu: the software was too big, too ambitious, and too slow to make a compelling product. Not only that, but users didn’t want virtual worlds nearly as much as everyone thought. Years later, and with 21st-century hardware, only Second Life has somewhat cracked the nut, and even their infrastructure is suffering pretty deeply right now (word has it).

Again, though, I met a number of super smart people who are now doing great things elsewhere, including Doug Crockford, moderately (ahem) well-known in Web 2.0 circles. Also Arturo Bejar, security ninja at Yahoo; Brian Marick, a major testing god; and Chip Morningstar and Randy Farmer, also now at Yahoo, who have a quite candid assessment of the whole era on THEIR blog. And I got to spend the better part of a year figuring out how to create a secure, capability-based protocol for virtual-world object transfer between untrusting hosts. Getting paid for that? It was really, really fun. Until the money ran out.

EC was really the end of my over-ambitious days. In 1998, after EC, I went on to work for Helium, a now-defunct consulting company founded by an old Protozoa buddy (Gever Tulley) and a number of his oldest friends. We did a bunch of work for Quokka.com (a truly classic dot-com flameout story — they picked the wrong sports to cover and created all their sites by hand, but we had fun doing Internet livecasting of sailing races and the 2000 Olympics!). Then Quokka folded in 2001, Helium reformed, and we went on like that until 2003.

Finally in 2003, I begain working for my current employer, Nimblefish. A number of my old Helium and Quokka friends joined me there, along with other great folks. I’ve been there for three and a half years now, an all-time record for my career, and we’ve got a working production system that just keeps getting better and better. Hibernate, Flex, and other modern tech drives our code; we’ve got real customers, and we’re growing; and we’ve got a clear strategy that we’re executing on at full bandwidth. I think I might have finally kicked the curse of over-ambition…..

…or maybe it’s just dormant! 🙂

I’ll close with the word that best captures how I feel about programming:

Onwards!

Hello folks. This blog has been too long in coming, due to many complicated reasons of my own self-inflicted devising. Better late than never. Enjoy!