robjsoftware.info

It’s amazing what a difference not commuting can make. Up until our move to Seattle in April, I’d been commuting an hour each way every day into San Francisco. That time on BART — without net access — was where all my blogging got done. Well, now I’m driving 15 minutes each way to work (modulo time to take my daughter to her school), and still coming down off the stress of moving, so I’ve been putting my energy elsewhere in the evenings. In any case, consider this a summer vacation for your RSS feed 🙂

Not that I’ve personally been vacationing — not at all! I’m finally getting up to some kind of reasonable cruising speed at work. It’s been a colossal re-education in .NET, C#, low-level Windows debugging, SQL data stack design, and about 50 different interesting research projects which I can’t really discuss — at least not yet. It’s very educational being on the list for all the different talks that happen inside Microsoft Research; there are a lot of different pies that those folks have their fingers in. The internal Windows developer talks are also very intriguing.

Now, this isn’t to say that everything is on the hush-hush. Some of the better publicly visible tidbits I’ve run across lately involve LINQ, the techniques for basically doing data-oriented metaprogramming in the .NET framework. I mentioned LINQ in my last post, but suffice to say that it’s a combination of a set of data-oriented query operators (that form a composable, monadic algebra), some syntactic extensions to allow more idiomatic SQL-like query expressions, and an internal syntax-tree representation that can be transformed into native query languages (including but not limited to SQL) and partially evaluated into high-performance serializers. Overall it’s a very well-thought-out structure with a lot of room for growth. To wit:

• DryadLINQ extends the LINQ operators to support MapReduce-like distributed cluster computation at scale.
• PLINQ and the ParallelFX framework are building up the .NET infrastructure for fine-grained parallelism and efficient use of multi-core for data-parallel problems.
• Matt Warren’s series on implementing a LINQ query provider is a great example of metaprogramming in practice — taking a C# abstract syntax tree describing a query, and transforming it into efficient SQL. This is the technique that is at the heart of LINQ. I’ve heard tell of F# projects that are using LINQ-like expressions to describe linear programming constraints declaratively — same metaprogramming paradigm, completely different domain.

All of this is exciting because you noble long-term readers (e.g. since December 2007) will know how interesting metaprogramming is to me. Microsoft really gets it on multiple levels, and has put it into production with LINQ. There’s a lot more on the horizon, too, and I’m eagerly waiting for each bit of it to reach RTM in some form so I can happily blog away about it!

Not only that, but I have a personal hacking project again. Matt Warren’s blogs (mentioned above) are a great example of how to implement metaprogramming with an imperative C# visitor-style API. But I find the code hard to read — it’s taking this lovely declarative representation and then transforming it with all of this intensely imperative, explicitly scheduled transformation code. It reminds me of the JastAdd project, which has evidently now reached public release. JastAdd creates a declarative framework for implementing language analyses and transformations. I want to hack on an F# implementation of the JastAdd paradigm, applied to LINQ to SQL transformations. It would be very interesting to see if it could lead to something significantly easier to maintain and reason about.

This is something that arguably is potentially relevant to work. So I am going to blog about it internally first, and then repost publicly a week or so later (on a per-post basis). If it gets to where it is interesting enough for Microsoft to want to privatize it, I’ll just have to find something else to blog publicly about! In any case, it’ll be fun to post about here for as long as it lasts 🙂