A blog about software – researching it, developing it, and contemplating its future.

Archive for the ‘Uncategorized’ Category

Holofunk Lives

with 3 comments

Allow me to demonstrate:

My last post was all about my big plans for making this thing, and now, here it is:  a Kinect-and-Wiimote-based live looping instrument, or soundspace, or synesthizer… not sure yet quite which.

It came together much faster than expected, under the very motivating mid-August realization that Beardyman (my inspiration) was playing Vancouver in mid-September, and that if I got a demo ready, maybe I could… show him!

Beardyman and Me?!

Lo and behold, after a frenzied and down-to-the-wire month of hacking, I had it working.  I recorded a (slightly NSFW) video, emailed it to him, and he saw it the day before his show.  It piqued his interest.

The next day there we were, ready to rock the world:

Beardyman is a down-to-earth guy, super friendly and gracious, exploding with ideas, and damn near as impressed with my work as I am with his.  (That last fact was a total shock, and a delightful one.)

And he tried it out!

(The mellow dude in the background is Ian, Darren’s tour manager.)

So, um, yeah, I’m pretty blown away right now.  As a huge bonus, I got to see his show that night, where he played some of the sickest and most intense drum and bass I’ve ever heard, making it all right there on the spot:

I can’t believe this all happened only a week ago.  It’s been dizzying and unforgettable.

So, Yes, Holofunk Is A Thing

Specifically, it is a thing right here: — warts and all.  You can download it and play with it if you like, and I quite encourage you to!  (You do need Visual Studio 2010 — this is a hacker’s project right now.)

It came together really amazingly rapidly.  XNA and C# were good rapid development choices, and the Kinect SDK and Wiimote libraries were both pretty much completely trouble-free.

But the single best technical choice was the BASS audio library.  I am very grateful to everyone who steered me in that direction.  I am using only the freeware version, but the questions I posted on their support forum got unbelievably prompt and complete responses from the two main developers.  If it weren’t for their help, there’s no way I would have been done on time.  I can’t recommend their project highly enough.

What It Is, And Isn’t

The open source site goes into much more detail, but basically, what you see above is what you currently get.  Beardyman and I had about a million ideas for what could be next.  Some of the ones I plan to experiment with over the next several months:

  • Pulling live video from the Kinect camera and animating it instead of just using colored circles.  (Imagine Monkey Jazz, live.)
  • Extracting the frequency and using it to colorize a sound trail (so you can literally paint with the loops).
  • Adding sound effects.
  • Chopping up and sub-looping your loops (possibly integrated with the video and/or sound trail).
Also, you may have noticed a glitch or two in some of the videos here.  That’s because while C# is fast to code in, it comes with a real problem, namely the garbage collector.  The BASS developers recommend writing core ASIO callback code in C++, purely to avoid interruptions from the .NET GC.  So that’s what I’ll be doing, and probably likewise with the video handling.

Another reason to do this is that the latest information on Win8 says that XNA, which I’m using as the game framework for Holofunk, is not going to be supported for the new tablet-style Metro apps.  Holofunk would make a great Metro app, so moving away from XNA for all media management is a good way to go.  I may wind up doing the whole thing in C++ just to avoid ever having to deal with random memory management interruptions from the runtime.

But I’m going to take a bit of a break for the next month or so.  If people are interested I will support all comers, but this was a big push to make this Beardyman awesomeness happen, and it’s time to personally ease up a bit 🙂

I would love to have some active collaborators on this thing.  There’s too much potential here for just one guy.  I also have a Holofunk Facebook group for anyone who wants to stay in touch with all Holofunkian doings.

However, before signing off:

Two More Tastes

This thing is so new and so raw that I feel I very much don’t have a handle on it yet, but here’s one more attempt.

And, finally, a guest appearance by my daughter:

Sophia is six.  I wanted Holofunk to be something she could play with and have fun.  And even that seems to have succeeded!


Written by robjellinghaus

2011/09/21 at 06:14

Posted in Holofunk, Uncategorized

Quiet, but busy

leave a comment »

Programmer blogs not uncommonly go dark for extended periods.  Those of us who work on fully open projects are probably the most loquacious, but that’s a minority.  Those of us who work on projects that aren’t publicly announced sometimes have a much harder time, because a lot of energy goes into awesome things that you just can’t tell the world about.  Yet.

That’s definitely been the case with me.  My quietude here recently is in inverse proportion to my productivity and energy at work.

But there’s one update I can give.  One of the best things about my current job is the caliber of people I work with.  Specifically, Rico Mariani became my boss last summer, and Neal Gafter is currently my mentor.  (Microsoft has a semi-official mentoring plan and I seem to have lucked out.)  Rico is quite well-known inside Microsoft, and Neal is quite well-known outside, and I’m doing my best to maximize my clue absorption bandwidth.

Best.  Learning.  Opportunities.  EVER!

Now I’m going to sign off (again!) and work on Hacking Project #2 during the 120 minutes remaining before sleep.  Like I said, I may be quiet, but I’m busy….

Written by robjellinghaus

2011/03/02 at 22:40

Posted in Uncategorized

Windows Phone 7 gets reactive

leave a comment »

OK so yes I am happy that all MS employees will get a Windows 7 phone.  I lost my last cellphone several months ago and have been limping along with a $20 TracFone that likes nothing more than connecting to random web sites and eating my meager share of prepaid minutes, all because I wanted to wait for the W7 phones.  So this is a nice bonus.

Meanwhile I have this Holofunk idea (see a couple of posts ago), and who knows what phones these days can do?  Also MS makes it really easy to get the W7 dev tools.

But the icing on this cake I’m considering baking is the Rx library for reactive programming, which ships as part of the SDK.  Check out the members of the IObservable interface.  If you aren’t familiar with the entire reactive concept, check out this and this.  I’m a firm believer in this way of structuring event-driven apps, and suddenly the prospect of writing a W7 phone app just got a whole lot more appealing.  Stay tuned!

Also, if anyone knows anything about sound recording through the W7Phone SDK, feel very free to share 🙂  It looks like this is the current contender… but… does it actually work?  On actual phones???

And now this post is becoming a liveblog of my surfing for info on this issue.  Danny Chen is the man on the scene in this very encouraging thread.  Looks like XNA is also the way to go; good to know.  And what’s more he is currently active on the XNA audio forum which could be a great resource.  Lots of people playing with audio on this, it sounds like!

Written by robjellinghaus

2010/07/26 at 21:58

Posted in Uncategorized

The Unitary Matrix

leave a comment »

A quickie post this time:

I mentioned a while back that the ACM’s monthly magazine Communications had taken a radical turn for the better.

Recently they had one of the most interesting articles I’ve yet read in any magazine, being a straightforward description of what quantum information is like to compute with.

The core idea is that a classical system is an N x M function, taking a state of size O(N) and converting it into a (potentially smaller) state of size M.  So the state evolution matrix of a classical system is an N x M matrix.

Quantum systems, on the other hand, are reversible.  This means that information is not lost in a quantum system; it is just transformed.  So the state evolution matrix of a quantum system is an N x N matrix, and moreover a unitary matrix that preserves magnitude (though not distribution).  Hence quantum computations are fundamentally affected by interference patterns; they gave an example of how a quantum random walk winds up with a very different distribution from a classical random walk.

It seems that the problems quantum computers are best at involve symmetries.  Since a quantum computer is a very efficient way of transforming a configuration, it makes intuitive sense that it would therefore be a fast way to determine symmetry.  Factoring large prime numbers is an example of finding such a symmetry (corresponding to the factorization itself).

It’s a fascinating article and explains all this much better than I am.  Reversible computing is one of those Holy Grails that I truly hope arrives some year in a fashion I am capable of programming.  It could be the real answer to energy issues in computing.  On the other hand, it could cost more energy to create a quantum system with the right unitary matrix in the first place than you get back when reversibly running it.  I very much hope we’ll find out in the next two or three decades.

Written by robjellinghaus

2010/06/29 at 10:17

Posted in Uncategorized

Glorious Possibilities #2: The Ultimate Parser

with one comment

So for various reasons I have been learning a lot about parsers lately.  The #1 thing I have learned is that parsing is all about tradeoffs.  The #2 thing I have learned is that the ultimate parsing environment does not yet exist.

Long-time readers of this blog (assuming I didn’t lose them all in the slightly bobbled switch to WordPress!) know that I’m very interested in growable languages and in extensible programming environments.  Parsing is at the heart of programming language design, since it brings meaning to the text we programmers write.  When writing extensible programming environments, it is important that your language be closed under composition, so extending your language’s grammar still has some meaning (even if it introduces more ambiguity).  Parsers, in this sense, want to be powerful.

On the other end of the spectrum, there are parsers for relatively straightforward formats such as XML or CSS.  These parsers are often running in resource-constrained environments (a cell phone browser) or are dealing with large volumes under high load (an enterprise messaging bus).  For these parsers, high performance is absolutely critical, sometimes in both memory and CPU.  Parsers, in this sense, want to be fast.

There is a very tricky tradeoff between power and speed throughout the parsing landscape.  As just one example, I’ve been experimenting with ANTLR lately.  It is, on many fronts, the best parsing toolkit available.  Terence Parr has done a bang-up job over many years of addressing difficult technical issues while building a very usable tool.  It is just purely wonderful that ANTLR’s generated code has line-by-line comments that come directly from the grammar that generated it.  ANTLRWorks is a very enjoyable tool for language design.  Finally, Terence’s books are immensely useful.

But even ANTLR has some surprises.  I generated CSharp2 code from the example CSS grammar on the ANTLR site.  Then I ran it and looked at its memory footprint with a profiler.  I was not surprised to see that there were an immense number of IToken objects being created — ANTLR’s generated lexer (as of version 3, anyway) allocates a new IToken object for each token parsed.  This alone is actually a really big performance problem for high-load parsers — allocation in the parser’s hot path is a big problem in a managed language.

There was a surprise, though, and not a good one.  There were close to 5 megabytes of int16[] arrays.  Where did those come from?  From the lexer DFAs, which are stored in static lexer fields and uncompressed when the lexer is loaded.  The compressed char[] representation of the lexer DFAs is tiny — certainly not megabytes.  So there is a shockingly high compression ratio here.  I have to question whether that grammar is really complex enough to motivate a multi-megabyte lexer DFA.  It seems that I am not the only person encountering surprisingly large lexers with ANTLR v3.  It looks like this can be worked around with a hand-written lexer, but wouldn’t it be wonderful if ANTLR itself were able to tune its lexer generation to achieve the same effect (trading off some speed, perhaps, to reduce DFA explosion)?

So there is room for performance improvement even on the most mature tools in the industry.  And on the other end of the spectrum, there are grammatical constructs that are impossible to express in purely context-free parsing frameworks like YACC.  I have been corresponding a bit with Yitzhak Mandelbaum at AT&T, subsequent to his excellent paper (with Trevor Jim and David Walker) on data-dependent parsing.  This has a lot of similarities with ANTLR’s semantic predicates, but seems to be perhaps more powerful and general (being embedded in a framework that is inherently scannerless).  However, being based on Earley parsing, it is hard for this method to gain competitive performance.  The method does support embedded “black-box” parsers, however.

The holy grail might be an environment with a powerful, compositional, potentially ambiguous top-level user language, and an implementation that could aggressively analyze sublanguages of the grammar and choose optimized implementations.  Grammar and language design is a balancing act between ambiguity and generality on the one hand, and efficiency on the other.  Different implementations have different tradeoffs — implementations supporting unambiguous grammars only will inevitably be faster than those that handle ambiguous parses; LR and LL algorithms both have their own idiosyncrasies with respect to the languages they can support; and so on.  So the ideal language environment would itself be extensible with different parsing algorithms that could themselves be composed.  That way, if all you have is a regular expression, you get tightly tuned inner loops and no backtracking; but if you have a highly ambiguous grammar, you get a backtracking and/or chart-parsing implementation, perhaps with lots of data dependencies.  The toolset should let you apply different algorithms to different parts of your language, and should support you in refactoring (for example, the left-corner transform) your grammar.

Incidentally, this last point is one reason why Parsing Expression Grammars are potentially problematic.  PEGs are a beautiful embedding of a declarative grammar in a functional programming language.  But in some sense most PEG implementations are too embedded — the only representation of the grammar’s structure is in the code itself, and all you can do with your PEG is to run it, not refactor or otherwise transform it.  (And let’s leave aside the issue of ordered choice potentially breaking your compositional intention.)  There is at least one endeavor to make PEG representations transformable, which is extremely interesting and worth following.

There is much more that could be said here — error recovery, for instance, is extremely tricky in many cases, and has tight interactions with the algorithm you are using.  Yet you still would like a declarative way to express your grammar’s error behavior, if possible. 

And what about low-level performance?  Which is faster:  a virtual object dispatch in your parser’s inner loop, or a colossal switch statement?  If you are doing code generation, the only answer is probably “generate both versions and then measure it on your specific grammar!”  There are so many tradeoffs here that drawing general conclusions may be nigh impossible.

The basic point I want to make is that parsing is very far from being a properly solved problem.  Tools like ANTLR point the way, but there is more still to be done.  And this is the glorious possibility:  building a parsing environment that is itself composable, and that is fully transparent to the developer at all times, letting the developer choose their implementation tradeoffs at all levels.

As Terence knows better than anyone, this alone could consume an entire career.  Or multiple such careers.  Still, I could see myself having a lot of fun building an F# implementation of this kind of transparent, composable parsing toolkit.  What a useful thing it would be!

Edit: And you know I will be following Terence’s work on ANTLR 4.  Also, this presentation of his on parser implementation is a great look at the state of the art in managed-language parser code generation. 

Edit #2:  I must also mention that ANTLR has soundly convinced me of the benefit of LL style: LL style is top-down recursive, which is a very natural way for most programmers to think.  Table-driven parsers, without good and scalable visualization, can be quite hard to debug and extend, but ANTLR’s generated code is a model of clarity.

Written by robjellinghaus

2010/05/04 at 22:51

Posted in Uncategorized

Glorious Possibilities #1: the Holofunk

with one comment

Yow!  Two postings in one night!

One reason for my reduced posting here is that I’ve realized this blog works a lot better if I have a personal hacking project. Hacking on something I can publicly talk about makes for much more interesting material. So I am glad that I have a surfeit of possibilities there, one or more of which will definitely be happening this year. The next three posts are devoted to these lovely options, any or all of which may or may not wind up happening, but which are nonetheless interesting in their own right.

Up today: the Holofunk.

A year or two ago I became aware of a British performer named Beardyman. I am an old school raver, and so I still follow the electronic / dance music scene. Beardyman won the best beatboxer in the UK prize two years in a row (a feat none else have accomplished), and these days he does one-man musical shows with his voice and a bevy of samplers and sequencers. The net result is that he is literally a one-man band. I find his work incredibly inspirational, as I like the electronic music scene and my voice is the only instrument I am much good with. So Beardyman is one inspiration.

Another inspiration of mine is my old college friend Leon Dewan. He and his cousin have created the Dewanatron, a unique device for making sounds never before conceived. I had the pleasure of joining them in a performance in Seattle recently (that would be me on the left at 3:15), and seeing them hacking their musical hardware was quite compelling. It’s fun to make weird sounds and bring order out of chaos out of order.

Now then, please integrate the above influences with the following facts:

1) There exists a piece of software named Ableton Live. (How existentially qualified of me!) This is a very powerful sequencing / synthesizing / sampling application used by many DJs and electronic music producers. Recently the Ableton people have done a lot of work to integrate Ableton Live with another audio product named Max, which is an exceptionally scriptable and flexible audio toolbox. The combined product is named Max for Live. There are existing C# libraries that allow Max, and hence Max for Live, to be controlled over a TCP connection.

2) 3D graphics cards are currently shipping; in fact, most mid-end to high-end PC

graphics cards can already drive a 3D display, and the hardware is about to become very widely available.

3) Microsoft this year, if their rumored timeframes hold true, is going to ship an attachment for the Xbox 360. This attachment is currently called Project Natal. It is essentially mass-market full-body video-based motion capture and 3D body tracking. The idea is that you just wave your arms and move your body and the thing watches you and can track you in realtime. (Actually, current rumors have it that the thing has about 100ms of lag, so you can’t expect instantaneous response out of it.) If it lives up to the hype — admittedly a big “if” — it really could be a step forward in interface technology.

So, let’s combine the artistic influences with the technical possibilities.

Let’s say that in late 2010 you could put together a reasonably fast PC with a 3D graphics card, an installation of Max for Live, and a PC-based Natal development kit. Let’s further say that Natal actually works, and that it is possible to — with enough experimentation and false starts — build an application that lets you directly manipulate 3D objects. (Hopefully Natal has at least some ability to detect finger position, or grabbing anything will be hard….)

If you could hook up a grab-and-drag Natal application to Max for Live, you could build a gesturally controlled 3D sound space. Add a microphone and whatever instruments you like, and make it really easy to script new gestures and new kinds of 3D sound-controlling objects.

It’d be a sonic holodeck. Punch the throbbing red sphere and start recording a new loop. Grab a sound and rip it in half, then push the two copies slightly out of tempo. Wave your arms around to change pitch, volume, modulation, or crossfade. I have no real idea what the detailed interface would be since I haven’t played with Natal for real yet.

But what I do know is that there is a real chance this could be an immensely entertaining thing for creative musicians to play with. I also know that I really want to play with it myself!

So that’s my first possible 2010 hacking project: the Holofunk. That’s what I’ll call it. If, of course, I ever finish it. If I do get somewhere with it, I will open source it, probably on Codeplex. And if anyone else is equally inspired by this, or otherwise has more ability to execute on it than I do (given my great day job and busy family life), by all means feel free! There is going to be some seriously fun art coming out of the techno scene in the next few years, and I can’t wait. I love the 21st century!

Written by robjellinghaus

2010/04/13 at 21:15

Posted in Uncategorized

Time keeps on slipping, slipping, slipping

leave a comment »

I originally thought that one posting a month was a reasonable and sustainable rate for this blog.

And actually, I still do think that. But I also think I’ve been less than effective at making it happen. Which is why I am going to make up for lost time. In 2009, I made… oh dear… five postings. That leaves a 2009 deficit of seven, plus three so far this year.

And it’s not as though I don’t appreciate the recent attention (for some definition of “recent”) that this blog has gotten. I do appreciate it. However, I don’t plan to make any similar posts in the foreseeable future. There is plenty of other stuff to talk about!

Working at Microsoft is an odd experience in some ways, because the company is so epically big that its output varies widely. So on the one hand there are plenty of failures to be embarrassed about, but on the other hand there are sometimes big wins that bring a kind of transitive corporate patriotic pride. For example, I never wanted a Windows phone before; no one had much good to say about them. But the bizarrely named Windows 7 Phone Series phones look great and I want one badly.

In fact, now I really need one, because I recently lost my only cell phone. So the question now is whether I can live without any cell phone whatsoever until these new goodies ship. They’d better ship in fall 2010 or I might have to go over somewhere and slap someone. Really, it is mostly catchup with the iPhone, but it is still nice to see a big company get with the program.

Next up: hacking resolutions for 2010!

Written by robjellinghaus

2010/04/13 at 21:13

Posted in Uncategorized

Good bye, Blogger

leave a comment »

If you’ve read this blog before, you may have noticed it now looks different.  I’ve moved from Blogger to WordPress, and based on experience so far, I’m not looking back.

Well, actually I guess I will, because it’s worth blogging about 🙂  I created this blog on Blogger almost three years ago.  The features Blogger had when I created this blog are almost exactly the features Blogger has today.

In some ways this was sort of fine.  I was used to the simple little textarea Blogger post window.  I was accustomed to the minimalist comments handling and the fairly clunky UI.

But then I took a hiatus from blogging, and when I came back, my most recent posts were fairly heavily spammed.  And that was not acceptable.

It’s odd, because Google is good at dealing with spam in Gmail.  Never had spam one there.  But both Google Groups and Blogger are definite spam victims at this point.  Google has a hit or miss record with its products, and Blogger has definitely been neglected.

WordPress, on the other hand, is widely considered to have the best spam control in the blogging industry.  I can already say that the blog editing UI is streets ahead of Blogger.  The content import from Blogger seems to have worked great.  So far I’m quite pleased, and willing to pay the $10 per year for domain mapping.

I am also working on a content revival for this blog.  Stay tuned!

Written by robjellinghaus

2010/03/10 at 07:59

Posted in Uncategorized

Because I can

with one comment

This is a list of worthy programmers. It makes me happy.

Unfortunately that is all I can say about this list at this time. Hopefully someday my disclosure may be fuller.

Written by robjellinghaus

2009/09/11 at 07:24

Posted in Uncategorized

Sex and the Clueless Coders

with one comment

I am very late to the “party” here, but I can’t help chiming in on the recent spate of programming conference presenters dropping bits of porn into their presentations. First it was the Golden Gate Ruby Conference in April, where a presentation on CouchDB (hmm, now I know what couch they meant!) included some racy pictures of mostly nekkid women. What’s more astonishing yet is that David Hennemeier Hanson, the Big Dog of the Ruby community, apparently thought this was all quite appropriate. I guess he wants the Ruby community to be all anti-corporate and rock-star. Or something.

And now as I was doing my remedial surfing to catch up on the story from April, I find that something similar went down (ahem) at a Minneapolis Flash conference, where the presenter did a big ol’ animation of all kinds of X-rated activities.

It’s funny, in a sad way, to see so many clueless male geeks sticking up for the L33t Rebels Busting Out The Pr0n. I mean, my God, guys, the entire Internet is filled with images of hotties of all genders, ages, species, and descriptions. Why do you feel the need to shove it in peoples’ faces at a geek conference?

I work for Microsoft now, which I suppose is an arch-corpocracy by the definitions of these running-wild-and-free, swinging-low coder cowboys (operative word being boys). Here at Microsoft we have this little thing called an HR policy. What it means is that if I wallpapered my desktop or office with pictures like the ones these zany idiots are slathering all over the place, I WOULD GET FIRED. And deservedly so. Because while sex is great, mixing it with work is guaranteed trouble for everyone.

It’s especially appalling how absolutely butt-ignorant many of these testosterone-poisoned hackers seem to be. I lived in San Francisco for a good twenty years, and I got to know a raft of feminist sex workers and general sex-positive people. And the one abiding principle that everyone thrived on was respect. Listening to what other people want and how other people feel — what a concept!!! And that’s exactly what all these indignant self-important immature coderboys are apparently utterly unable to do, when confronted with the thoroughly understandable discomfort of the women and the more enlightened men who were present at these code talks that turned into bad peep shows.

I love programming. I love computer science. The world of software has unlimited potential. And the lack of women in this field is beyond tragic. This summer I’m actually managing a female intern in our group; she is rocking the code in a serious way and I’m honored to get to be her manager. Looking at this unbelievable travesty of civilized behavior at these conferences, I’m just really, really glad we’re not on any Ruby or Flash projects, and I’m also glad we’re in a Big Boring Corporation, because not only are we working on things that are astonishingly cool despite our corporate overlords, we are protected from the kind of juvenile bullshit that seems to have infected the Ruby world.

We want MORE women in this field, not less — but if the numbskull, sexist Rubyists and their ilk have their way, all we’ll have is stupid guys with more balls than brains. And that’s bad for everyone. KEEP IT IN YOUR DAMN PANTS, IDIOTS! And do your pr0n surfing in private!

Edit: Here’s the big thread on the Ruby debacle, with the presenter in question chiming in. Now you know which side I’m on….

More editing: Here’s Martin Fowler’s excellent summary of the debacle. And here’s DHH himself — scroll down to April 27. Kind of funny how he seems to think that enjoying the movie Pulp Fiction and putting porn into a technical conference are somehow related. Dude, don’t be so proud of being R-rated — I’m sometimes very X-rated myself, but you won’t find out about it here, because I know better than to get my boundaries mixed up! Good fences make good neighbors.

Written by robjellinghaus

2009/07/07 at 07:24

Posted in Uncategorized