A blog about software – researching it, developing it, and contemplating its future.

Archive for the ‘Uncategorized’ Category


with one comment

Life has been busy in Holofunk land!  First, a new video:

While my singing needs work at one point, the overall concept is finally actually there:  you can layer things in a reasonably tight way, and you can tweak your sounds in groups.

Holofunk Jam, June 23rd

I have no shortage of feature ideas, and I’m going to be hacking on this thing for the foreseeable future, but in the near term:  on June 23rd I’m organizing a “Holofunk Jam” at the Seattle home of some very generous friends.  I’m going to set up Holofunk, demo it, ask anyone & everyone to try it, and hopefully see various gadgets, loopers, etc. that people bring over.  It would be amazing if it turned into a free-form electronica jam session of some kind!  If this sounds interesting to you, drop me a line.

Demoing Holofunk

There have been two public Holofunk demos since my last post, both of them enjoyable and educational.

Microsoft had a Hardware Summit, including the “science fair” I mentioned in my last post.  I wound up winning the “Golden Volcano” award in the Kinect category.  GO ME!  This in practice meant a small wooden laser-etched cube:

This was rather like coming in third out of about eight Kinect projects, which is actually not bad as the competition was quite impressive — e.g. an India team doing Kinect sign language recognition.  The big lesson from this event:  if someone is really interested in your project, don’t just give them your info, get their info too.  I would love to follow up with some of the people who came, but they seem unfindable!

Then, last weekend, the Maker Faire did indeed happen — and shame on me for not updating this blog in realtime with it.  I was picked as a presenter, and things went quite well, no mishaps to speak of.  In fact, I opened with a little riff, aand when it ended I got spontaneous applause!  Unexpected and appreciated.  (They also applauded at the end.)

I videoed it, but did not record the PA system, which was a terrible failure on my part; all the camera picked up was the roar of the people hobnobbing around the booths in the presentation room.  Still, it was a lot of fun and people seemed to like it.

My kids had a great time at the faire, too.  Here they are watching (and hearing) a record player, for the very first time in their lives:

True children of the 21st century 🙂

Coming Soon

I’ll be making another source drop to soon — trying to keep things up to date.  And the next features on the list:

  • effect selection / menuing
  • panning
  • volume
  • reverb
  • delay
  • effect recording
  • VST support

Well, maybe not that last one quite yet, but we’ll see.  And of course practice, practice, practice!


Written by robjellinghaus

2012/06/09 at 00:20

Posted in Holofunk, Uncategorized

Science fair time!

leave a comment »

Holofunk has been externally hibernating since last September; first I took a few months off just on general principles, and since then I’ve been hacking on the down-low.  In that time I’ve fixed Holofunk’s time sync issue (thanks again to the stupendous free support from the BASS Audio library guys).  I’ve added a number of visual cues to help people follow what’s happening, including beat meters to show how many beats long each track is, and better track length setting — now tracks can only be 1, 2, or a multiple of 4 beats long, making it easy to line things up.  Generally I’m in a very satisfying hacking groove now.

And today Holofunk re-emerges into the public eye — I’m demoing at a Microsoft internal event dubbed the Science Fair, coinciding with Microsoft’s annual Hardware Summit.  Root for me to win a prize if you have any good karma to spare today 🙂  I’ll post again in a day or two with an update on how it went.

I’ve also applied to be a speaker at the Seattle Mini Maker Faire the first weekend in June — will find out about that within a week.  If that happens, then I’ll spread the word as to exactly when I’ll be presenting!

Written by robjellinghaus

2012/05/10 at 06:33

Posted in Holofunk, Uncategorized

Holofunk Lives

with 3 comments

Allow me to demonstrate:

My last post was all about my big plans for making this thing, and now, here it is:  a Kinect-and-Wiimote-based live looping instrument, or soundspace, or synesthizer… not sure yet quite which.

It came together much faster than expected, under the very motivating mid-August realization that Beardyman (my inspiration) was playing Vancouver in mid-September, and that if I got a demo ready, maybe I could… show him!

Beardyman and Me?!

Lo and behold, after a frenzied and down-to-the-wire month of hacking, I had it working.  I recorded a (slightly NSFW) video, emailed it to him, and he saw it the day before his show.  It piqued his interest.

The next day there we were, ready to rock the world:

Beardyman is a down-to-earth guy, super friendly and gracious, exploding with ideas, and damn near as impressed with my work as I am with his.  (That last fact was a total shock, and a delightful one.)

And he tried it out!

(The mellow dude in the background is Ian, Darren’s tour manager.)

So, um, yeah, I’m pretty blown away right now.  As a huge bonus, I got to see his show that night, where he played some of the sickest and most intense drum and bass I’ve ever heard, making it all right there on the spot:

I can’t believe this all happened only a week ago.  It’s been dizzying and unforgettable.

So, Yes, Holofunk Is A Thing

Specifically, it is a thing right here: — warts and all.  You can download it and play with it if you like, and I quite encourage you to!  (You do need Visual Studio 2010 — this is a hacker’s project right now.)

It came together really amazingly rapidly.  XNA and C# were good rapid development choices, and the Kinect SDK and Wiimote libraries were both pretty much completely trouble-free.

But the single best technical choice was the BASS audio library.  I am very grateful to everyone who steered me in that direction.  I am using only the freeware version, but the questions I posted on their support forum got unbelievably prompt and complete responses from the two main developers.  If it weren’t for their help, there’s no way I would have been done on time.  I can’t recommend their project highly enough.

What It Is, And Isn’t

The open source site goes into much more detail, but basically, what you see above is what you currently get.  Beardyman and I had about a million ideas for what could be next.  Some of the ones I plan to experiment with over the next several months:

  • Pulling live video from the Kinect camera and animating it instead of just using colored circles.  (Imagine Monkey Jazz, live.)
  • Extracting the frequency and using it to colorize a sound trail (so you can literally paint with the loops).
  • Adding sound effects.
  • Chopping up and sub-looping your loops (possibly integrated with the video and/or sound trail).
Also, you may have noticed a glitch or two in some of the videos here.  That’s because while C# is fast to code in, it comes with a real problem, namely the garbage collector.  The BASS developers recommend writing core ASIO callback code in C++, purely to avoid interruptions from the .NET GC.  So that’s what I’ll be doing, and probably likewise with the video handling.

Another reason to do this is that the latest information on Win8 says that XNA, which I’m using as the game framework for Holofunk, is not going to be supported for the new tablet-style Metro apps.  Holofunk would make a great Metro app, so moving away from XNA for all media management is a good way to go.  I may wind up doing the whole thing in C++ just to avoid ever having to deal with random memory management interruptions from the runtime.

But I’m going to take a bit of a break for the next month or so.  If people are interested I will support all comers, but this was a big push to make this Beardyman awesomeness happen, and it’s time to personally ease up a bit 🙂

I would love to have some active collaborators on this thing.  There’s too much potential here for just one guy.  I also have a Holofunk Facebook group for anyone who wants to stay in touch with all Holofunkian doings.

However, before signing off:

Two More Tastes

This thing is so new and so raw that I feel I very much don’t have a handle on it yet, but here’s one more attempt.

And, finally, a guest appearance by my daughter:

Sophia is six.  I wanted Holofunk to be something she could play with and have fun.  And even that seems to have succeeded!

Written by robjellinghaus

2011/09/21 at 06:14

Posted in Holofunk, Uncategorized

Quiet, but busy

leave a comment »

Programmer blogs not uncommonly go dark for extended periods.  Those of us who work on fully open projects are probably the most loquacious, but that’s a minority.  Those of us who work on projects that aren’t publicly announced sometimes have a much harder time, because a lot of energy goes into awesome things that you just can’t tell the world about.  Yet.

That’s definitely been the case with me.  My quietude here recently is in inverse proportion to my productivity and energy at work.

But there’s one update I can give.  One of the best things about my current job is the caliber of people I work with.  Specifically, Rico Mariani became my boss last summer, and Neal Gafter is currently my mentor.  (Microsoft has a semi-official mentoring plan and I seem to have lucked out.)  Rico is quite well-known inside Microsoft, and Neal is quite well-known outside, and I’m doing my best to maximize my clue absorption bandwidth.

Best.  Learning.  Opportunities.  EVER!

Now I’m going to sign off (again!) and work on Hacking Project #2 during the 120 minutes remaining before sleep.  Like I said, I may be quiet, but I’m busy….

Written by robjellinghaus

2011/03/02 at 22:40

Posted in Uncategorized

Windows Phone 7 gets reactive

leave a comment »

OK so yes I am happy that all MS employees will get a Windows 7 phone.  I lost my last cellphone several months ago and have been limping along with a $20 TracFone that likes nothing more than connecting to random web sites and eating my meager share of prepaid minutes, all because I wanted to wait for the W7 phones.  So this is a nice bonus.

Meanwhile I have this Holofunk idea (see a couple of posts ago), and who knows what phones these days can do?  Also MS makes it really easy to get the W7 dev tools.

But the icing on this cake I’m considering baking is the Rx library for reactive programming, which ships as part of the SDK.  Check out the members of the IObservable interface.  If you aren’t familiar with the entire reactive concept, check out this and this.  I’m a firm believer in this way of structuring event-driven apps, and suddenly the prospect of writing a W7 phone app just got a whole lot more appealing.  Stay tuned!

Also, if anyone knows anything about sound recording through the W7Phone SDK, feel very free to share 🙂  It looks like this is the current contender… but… does it actually work?  On actual phones???

And now this post is becoming a liveblog of my surfing for info on this issue.  Danny Chen is the man on the scene in this very encouraging thread.  Looks like XNA is also the way to go; good to know.  And what’s more he is currently active on the XNA audio forum which could be a great resource.  Lots of people playing with audio on this, it sounds like!

Written by robjellinghaus

2010/07/26 at 21:58

Posted in Uncategorized

The Unitary Matrix

leave a comment »

A quickie post this time:

I mentioned a while back that the ACM’s monthly magazine Communications had taken a radical turn for the better.

Recently they had one of the most interesting articles I’ve yet read in any magazine, being a straightforward description of what quantum information is like to compute with.

The core idea is that a classical system is an N x M function, taking a state of size O(N) and converting it into a (potentially smaller) state of size M.  So the state evolution matrix of a classical system is an N x M matrix.

Quantum systems, on the other hand, are reversible.  This means that information is not lost in a quantum system; it is just transformed.  So the state evolution matrix of a quantum system is an N x N matrix, and moreover a unitary matrix that preserves magnitude (though not distribution).  Hence quantum computations are fundamentally affected by interference patterns; they gave an example of how a quantum random walk winds up with a very different distribution from a classical random walk.

It seems that the problems quantum computers are best at involve symmetries.  Since a quantum computer is a very efficient way of transforming a configuration, it makes intuitive sense that it would therefore be a fast way to determine symmetry.  Factoring large prime numbers is an example of finding such a symmetry (corresponding to the factorization itself).

It’s a fascinating article and explains all this much better than I am.  Reversible computing is one of those Holy Grails that I truly hope arrives some year in a fashion I am capable of programming.  It could be the real answer to energy issues in computing.  On the other hand, it could cost more energy to create a quantum system with the right unitary matrix in the first place than you get back when reversibly running it.  I very much hope we’ll find out in the next two or three decades.

Written by robjellinghaus

2010/06/29 at 10:17

Posted in Uncategorized

Glorious Possibilities #2: The Ultimate Parser

with one comment

So for various reasons I have been learning a lot about parsers lately.  The #1 thing I have learned is that parsing is all about tradeoffs.  The #2 thing I have learned is that the ultimate parsing environment does not yet exist.

Long-time readers of this blog (assuming I didn’t lose them all in the slightly bobbled switch to WordPress!) know that I’m very interested in growable languages and in extensible programming environments.  Parsing is at the heart of programming language design, since it brings meaning to the text we programmers write.  When writing extensible programming environments, it is important that your language be closed under composition, so extending your language’s grammar still has some meaning (even if it introduces more ambiguity).  Parsers, in this sense, want to be powerful.

On the other end of the spectrum, there are parsers for relatively straightforward formats such as XML or CSS.  These parsers are often running in resource-constrained environments (a cell phone browser) or are dealing with large volumes under high load (an enterprise messaging bus).  For these parsers, high performance is absolutely critical, sometimes in both memory and CPU.  Parsers, in this sense, want to be fast.

There is a very tricky tradeoff between power and speed throughout the parsing landscape.  As just one example, I’ve been experimenting with ANTLR lately.  It is, on many fronts, the best parsing toolkit available.  Terence Parr has done a bang-up job over many years of addressing difficult technical issues while building a very usable tool.  It is just purely wonderful that ANTLR’s generated code has line-by-line comments that come directly from the grammar that generated it.  ANTLRWorks is a very enjoyable tool for language design.  Finally, Terence’s books are immensely useful.

But even ANTLR has some surprises.  I generated CSharp2 code from the example CSS grammar on the ANTLR site.  Then I ran it and looked at its memory footprint with a profiler.  I was not surprised to see that there were an immense number of IToken objects being created — ANTLR’s generated lexer (as of version 3, anyway) allocates a new IToken object for each token parsed.  This alone is actually a really big performance problem for high-load parsers — allocation in the parser’s hot path is a big problem in a managed language.

There was a surprise, though, and not a good one.  There were close to 5 megabytes of int16[] arrays.  Where did those come from?  From the lexer DFAs, which are stored in static lexer fields and uncompressed when the lexer is loaded.  The compressed char[] representation of the lexer DFAs is tiny — certainly not megabytes.  So there is a shockingly high compression ratio here.  I have to question whether that grammar is really complex enough to motivate a multi-megabyte lexer DFA.  It seems that I am not the only person encountering surprisingly large lexers with ANTLR v3.  It looks like this can be worked around with a hand-written lexer, but wouldn’t it be wonderful if ANTLR itself were able to tune its lexer generation to achieve the same effect (trading off some speed, perhaps, to reduce DFA explosion)?

So there is room for performance improvement even on the most mature tools in the industry.  And on the other end of the spectrum, there are grammatical constructs that are impossible to express in purely context-free parsing frameworks like YACC.  I have been corresponding a bit with Yitzhak Mandelbaum at AT&T, subsequent to his excellent paper (with Trevor Jim and David Walker) on data-dependent parsing.  This has a lot of similarities with ANTLR’s semantic predicates, but seems to be perhaps more powerful and general (being embedded in a framework that is inherently scannerless).  However, being based on Earley parsing, it is hard for this method to gain competitive performance.  The method does support embedded “black-box” parsers, however.

The holy grail might be an environment with a powerful, compositional, potentially ambiguous top-level user language, and an implementation that could aggressively analyze sublanguages of the grammar and choose optimized implementations.  Grammar and language design is a balancing act between ambiguity and generality on the one hand, and efficiency on the other.  Different implementations have different tradeoffs — implementations supporting unambiguous grammars only will inevitably be faster than those that handle ambiguous parses; LR and LL algorithms both have their own idiosyncrasies with respect to the languages they can support; and so on.  So the ideal language environment would itself be extensible with different parsing algorithms that could themselves be composed.  That way, if all you have is a regular expression, you get tightly tuned inner loops and no backtracking; but if you have a highly ambiguous grammar, you get a backtracking and/or chart-parsing implementation, perhaps with lots of data dependencies.  The toolset should let you apply different algorithms to different parts of your language, and should support you in refactoring (for example, the left-corner transform) your grammar.

Incidentally, this last point is one reason why Parsing Expression Grammars are potentially problematic.  PEGs are a beautiful embedding of a declarative grammar in a functional programming language.  But in some sense most PEG implementations are too embedded — the only representation of the grammar’s structure is in the code itself, and all you can do with your PEG is to run it, not refactor or otherwise transform it.  (And let’s leave aside the issue of ordered choice potentially breaking your compositional intention.)  There is at least one endeavor to make PEG representations transformable, which is extremely interesting and worth following.

There is much more that could be said here — error recovery, for instance, is extremely tricky in many cases, and has tight interactions with the algorithm you are using.  Yet you still would like a declarative way to express your grammar’s error behavior, if possible. 

And what about low-level performance?  Which is faster:  a virtual object dispatch in your parser’s inner loop, or a colossal switch statement?  If you are doing code generation, the only answer is probably “generate both versions and then measure it on your specific grammar!”  There are so many tradeoffs here that drawing general conclusions may be nigh impossible.

The basic point I want to make is that parsing is very far from being a properly solved problem.  Tools like ANTLR point the way, but there is more still to be done.  And this is the glorious possibility:  building a parsing environment that is itself composable, and that is fully transparent to the developer at all times, letting the developer choose their implementation tradeoffs at all levels.

As Terence knows better than anyone, this alone could consume an entire career.  Or multiple such careers.  Still, I could see myself having a lot of fun building an F# implementation of this kind of transparent, composable parsing toolkit.  What a useful thing it would be!

Edit: And you know I will be following Terence’s work on ANTLR 4.  Also, this presentation of his on parser implementation is a great look at the state of the art in managed-language parser code generation. 

Edit #2:  I must also mention that ANTLR has soundly convinced me of the benefit of LL style: LL style is top-down recursive, which is a very natural way for most programmers to think.  Table-driven parsers, without good and scalable visualization, can be quite hard to debug and extend, but ANTLR’s generated code is a model of clarity.

Written by robjellinghaus

2010/05/04 at 22:51

Posted in Uncategorized