A blog about software – researching it, developing it, and contemplating its future.

Archive for the ‘Uncategorized’ Category

SlimDX vs. SharpDX

with 11 comments

Phew!  Been very busy around here.  The Holofunk Jam, mentioned last post, went very well — met a few talented local loopers who gave me invaluable hands-on advice.  Demoed to the Kinect for Windows team and got some good feedback there.  My sister has requested a Holofunk performance at her wedding in Boston near the end of August, and before that, the Microsoft Garage team has twisted my arm to give another public demo on August 16th.  Plus I had my tenth wedding anniversary with my wife last weekend.  Life is full, full, FULL!  And I’m in no way whatsoever complaining.

Time To Put Up, Or Else To Shut Up

One piece of feedback I’ve gotten consistently is that darn near everyone is skeptical that this thing can really be useful for full-on performance.  “It’s a fun Kinect-y toy,” many say, “but it needs a lot of work before you can take it on stage.”  This is emerging as the central challenge of this project: can I get it to the point where I can credibly rock a room with it?  If I personally can’t use it to funk out in an undeniable and audience-connected manner, it’s for damn sure no one else will be able to either.

So it’s time to focus on performance features for the software, and improved beatboxing and looping skills for me!

The number one performance feature it needs is dual monitor support.  Right now, when you’re using Holofunk, you’re facing a screen on which your image is projected.  The Kinect is under the screen, facing you, and the screen shows what the Kinect sees.

This is standard Kinect videogame setup — you are effectively looking at your mirrored video image, which moves as you do.  It’s great… if you’re the only one playing.

But if you have an audience, then the audience is looking at your back, and you’re all (you and the audience) looking at the projected screen.


No solo performer wants their back to the audience.

So what I need is dual screen support.  I should be able to have Holofunk on my laptop.  I face the audience; the laptop is between me and the audience, facing me; I’m watching the laptop screen and Holofunking on it.  The Kinect is sitting by the laptop, and the laptop is putting out a mirror-reversed image for the projection screen behind me, which the audience is watching.

Like this:

With that setup, I can make eye contact with the audience while still driving Holofunk, and the audience can still see what I’m doing with Holofunk.

So, that’s the number one feature… probably the only major feature I’ll be adding before next month’s demos.

The question is, how?

XNA No More

Right now Holofunk uses the XNA C# graphics library from Microsoft.  Problem is, this seems defunct; it is stuck on DirectX 9 (a several-year-old graphics API at this point), and there is no indication it will ever be made available for Windows 8 Metro.

I looked into porting Holofunk to C++.  It was terrifying.  I’ll be sticking with C#, thanks.  But not only is XNA a dead end, it doesn’t support multiple displays!  You get only one game window.

So I’ve got to switch sooner rather than later.  The two big contenders in the C# world are SlimDX and SharpDX.

In a nutshell:  SlimDX has been around for longer, and has significantly better documentation.  SharpDX is more up-to-date (it already has Windows 8 support, unlike SlimDX), and is “closer to the metal” (it’s more consistently generated directly from the DirectX C++ API definitions).

As always in the open source world, one of the first things to check — beyond “do the samples compile?” and “is there any API documentation?” — is how many commits have been made recently to the projects’ source trees.

In the SlimDX case, there was a flurry of activity back in March, and since then there has been very little activity at all.  In the SharpDX case, the developer is an animal and is frenetically committing almost every day.

SharpDX’s most recent release is from last month.  SlimDX’s is from January.

Two of the main SlimDX developers have moved on (as explicitly stated in their blogs), and the third seems AWOL.

Finally, I found this thread about possible directions for SlimDX 2, and it doesn’t seem that anyone is actively carrying the torch.

So, SharpDX wins from a support perspective.  The problem for me is, it looks like a lot of DirectX boilerplate compared to XNA.

I just, though, turned up a reference to this other project ANX — an XNA-compatible API wrapper around SharpDX.  That looks just about perfect for me.  So I will be investigating ANX on top of SharpDX first; if that falls through, I’ll go just with SharpDX alone.

This is daunting simply because it’s always a bit of a drag to switch to a new framework — they all have learning curves, and XNA’s was easy, but SharpDX’s won’t be.  So I have to psych myself up for it a bit.  The good news, though, is once I have a more modern API under the hood, I can start doing crazy things like realtime video recording and video texture playback… that’s a 2013 feature at the earliest, by the way 🙂

Written by robjellinghaus

2012/07/18 at 23:54

Posted in Holofunk, Uncategorized


with one comment

Life has been busy in Holofunk land!  First, a new video:

While my singing needs work at one point, the overall concept is finally actually there:  you can layer things in a reasonably tight way, and you can tweak your sounds in groups.

Holofunk Jam, June 23rd

I have no shortage of feature ideas, and I’m going to be hacking on this thing for the foreseeable future, but in the near term:  on June 23rd I’m organizing a “Holofunk Jam” at the Seattle home of some very generous friends.  I’m going to set up Holofunk, demo it, ask anyone & everyone to try it, and hopefully see various gadgets, loopers, etc. that people bring over.  It would be amazing if it turned into a free-form electronica jam session of some kind!  If this sounds interesting to you, drop me a line.

Demoing Holofunk

There have been two public Holofunk demos since my last post, both of them enjoyable and educational.

Microsoft had a Hardware Summit, including the “science fair” I mentioned in my last post.  I wound up winning the “Golden Volcano” award in the Kinect category.  GO ME!  This in practice meant a small wooden laser-etched cube:

This was rather like coming in third out of about eight Kinect projects, which is actually not bad as the competition was quite impressive — e.g. an India team doing Kinect sign language recognition.  The big lesson from this event:  if someone is really interested in your project, don’t just give them your info, get their info too.  I would love to follow up with some of the people who came, but they seem unfindable!

Then, last weekend, the Maker Faire did indeed happen — and shame on me for not updating this blog in realtime with it.  I was picked as a presenter, and things went quite well, no mishaps to speak of.  In fact, I opened with a little riff, aand when it ended I got spontaneous applause!  Unexpected and appreciated.  (They also applauded at the end.)

I videoed it, but did not record the PA system, which was a terrible failure on my part; all the camera picked up was the roar of the people hobnobbing around the booths in the presentation room.  Still, it was a lot of fun and people seemed to like it.

My kids had a great time at the faire, too.  Here they are watching (and hearing) a record player, for the very first time in their lives:

True children of the 21st century 🙂

Coming Soon

I’ll be making another source drop to soon — trying to keep things up to date.  And the next features on the list:

  • effect selection / menuing
  • panning
  • volume
  • reverb
  • delay
  • effect recording
  • VST support

Well, maybe not that last one quite yet, but we’ll see.  And of course practice, practice, practice!

Written by robjellinghaus

2012/06/09 at 00:20

Posted in Holofunk, Uncategorized

Science fair time!

leave a comment »

Holofunk has been externally hibernating since last September; first I took a few months off just on general principles, and since then I’ve been hacking on the down-low.  In that time I’ve fixed Holofunk’s time sync issue (thanks again to the stupendous free support from the BASS Audio library guys).  I’ve added a number of visual cues to help people follow what’s happening, including beat meters to show how many beats long each track is, and better track length setting — now tracks can only be 1, 2, or a multiple of 4 beats long, making it easy to line things up.  Generally I’m in a very satisfying hacking groove now.

And today Holofunk re-emerges into the public eye — I’m demoing at a Microsoft internal event dubbed the Science Fair, coinciding with Microsoft’s annual Hardware Summit.  Root for me to win a prize if you have any good karma to spare today 🙂  I’ll post again in a day or two with an update on how it went.

I’ve also applied to be a speaker at the Seattle Mini Maker Faire the first weekend in June — will find out about that within a week.  If that happens, then I’ll spread the word as to exactly when I’ll be presenting!

Written by robjellinghaus

2012/05/10 at 06:33

Posted in Holofunk, Uncategorized

Holofunk Lives

with 3 comments

Allow me to demonstrate:

My last post was all about my big plans for making this thing, and now, here it is:  a Kinect-and-Wiimote-based live looping instrument, or soundspace, or synesthizer… not sure yet quite which.

It came together much faster than expected, under the very motivating mid-August realization that Beardyman (my inspiration) was playing Vancouver in mid-September, and that if I got a demo ready, maybe I could… show him!

Beardyman and Me?!

Lo and behold, after a frenzied and down-to-the-wire month of hacking, I had it working.  I recorded a (slightly NSFW) video, emailed it to him, and he saw it the day before his show.  It piqued his interest.

The next day there we were, ready to rock the world:

Beardyman is a down-to-earth guy, super friendly and gracious, exploding with ideas, and damn near as impressed with my work as I am with his.  (That last fact was a total shock, and a delightful one.)

And he tried it out!

(The mellow dude in the background is Ian, Darren’s tour manager.)

So, um, yeah, I’m pretty blown away right now.  As a huge bonus, I got to see his show that night, where he played some of the sickest and most intense drum and bass I’ve ever heard, making it all right there on the spot:

I can’t believe this all happened only a week ago.  It’s been dizzying and unforgettable.

So, Yes, Holofunk Is A Thing

Specifically, it is a thing right here: — warts and all.  You can download it and play with it if you like, and I quite encourage you to!  (You do need Visual Studio 2010 — this is a hacker’s project right now.)

It came together really amazingly rapidly.  XNA and C# were good rapid development choices, and the Kinect SDK and Wiimote libraries were both pretty much completely trouble-free.

But the single best technical choice was the BASS audio library.  I am very grateful to everyone who steered me in that direction.  I am using only the freeware version, but the questions I posted on their support forum got unbelievably prompt and complete responses from the two main developers.  If it weren’t for their help, there’s no way I would have been done on time.  I can’t recommend their project highly enough.

What It Is, And Isn’t

The open source site goes into much more detail, but basically, what you see above is what you currently get.  Beardyman and I had about a million ideas for what could be next.  Some of the ones I plan to experiment with over the next several months:

  • Pulling live video from the Kinect camera and animating it instead of just using colored circles.  (Imagine Monkey Jazz, live.)
  • Extracting the frequency and using it to colorize a sound trail (so you can literally paint with the loops).
  • Adding sound effects.
  • Chopping up and sub-looping your loops (possibly integrated with the video and/or sound trail).
Also, you may have noticed a glitch or two in some of the videos here.  That’s because while C# is fast to code in, it comes with a real problem, namely the garbage collector.  The BASS developers recommend writing core ASIO callback code in C++, purely to avoid interruptions from the .NET GC.  So that’s what I’ll be doing, and probably likewise with the video handling.

Another reason to do this is that the latest information on Win8 says that XNA, which I’m using as the game framework for Holofunk, is not going to be supported for the new tablet-style Metro apps.  Holofunk would make a great Metro app, so moving away from XNA for all media management is a good way to go.  I may wind up doing the whole thing in C++ just to avoid ever having to deal with random memory management interruptions from the runtime.

But I’m going to take a bit of a break for the next month or so.  If people are interested I will support all comers, but this was a big push to make this Beardyman awesomeness happen, and it’s time to personally ease up a bit 🙂

I would love to have some active collaborators on this thing.  There’s too much potential here for just one guy.  I also have a Holofunk Facebook group for anyone who wants to stay in touch with all Holofunkian doings.

However, before signing off:

Two More Tastes

This thing is so new and so raw that I feel I very much don’t have a handle on it yet, but here’s one more attempt.

And, finally, a guest appearance by my daughter:

Sophia is six.  I wanted Holofunk to be something she could play with and have fun.  And even that seems to have succeeded!

Written by robjellinghaus

2011/09/21 at 06:14

Posted in Holofunk, Uncategorized

Quiet, but busy

leave a comment »

Programmer blogs not uncommonly go dark for extended periods.  Those of us who work on fully open projects are probably the most loquacious, but that’s a minority.  Those of us who work on projects that aren’t publicly announced sometimes have a much harder time, because a lot of energy goes into awesome things that you just can’t tell the world about.  Yet.

That’s definitely been the case with me.  My quietude here recently is in inverse proportion to my productivity and energy at work.

But there’s one update I can give.  One of the best things about my current job is the caliber of people I work with.  Specifically, Rico Mariani became my boss last summer, and Neal Gafter is currently my mentor.  (Microsoft has a semi-official mentoring plan and I seem to have lucked out.)  Rico is quite well-known inside Microsoft, and Neal is quite well-known outside, and I’m doing my best to maximize my clue absorption bandwidth.

Best.  Learning.  Opportunities.  EVER!

Now I’m going to sign off (again!) and work on Hacking Project #2 during the 120 minutes remaining before sleep.  Like I said, I may be quiet, but I’m busy….

Written by robjellinghaus

2011/03/02 at 22:40

Posted in Uncategorized

Windows Phone 7 gets reactive

leave a comment »

OK so yes I am happy that all MS employees will get a Windows 7 phone.  I lost my last cellphone several months ago and have been limping along with a $20 TracFone that likes nothing more than connecting to random web sites and eating my meager share of prepaid minutes, all because I wanted to wait for the W7 phones.  So this is a nice bonus.

Meanwhile I have this Holofunk idea (see a couple of posts ago), and who knows what phones these days can do?  Also MS makes it really easy to get the W7 dev tools.

But the icing on this cake I’m considering baking is the Rx library for reactive programming, which ships as part of the SDK.  Check out the members of the IObservable interface.  If you aren’t familiar with the entire reactive concept, check out this and this.  I’m a firm believer in this way of structuring event-driven apps, and suddenly the prospect of writing a W7 phone app just got a whole lot more appealing.  Stay tuned!

Also, if anyone knows anything about sound recording through the W7Phone SDK, feel very free to share 🙂  It looks like this is the current contender… but… does it actually work?  On actual phones???

And now this post is becoming a liveblog of my surfing for info on this issue.  Danny Chen is the man on the scene in this very encouraging thread.  Looks like XNA is also the way to go; good to know.  And what’s more he is currently active on the XNA audio forum which could be a great resource.  Lots of people playing with audio on this, it sounds like!

Written by robjellinghaus

2010/07/26 at 21:58

Posted in Uncategorized

The Unitary Matrix

leave a comment »

A quickie post this time:

I mentioned a while back that the ACM’s monthly magazine Communications had taken a radical turn for the better.

Recently they had one of the most interesting articles I’ve yet read in any magazine, being a straightforward description of what quantum information is like to compute with.

The core idea is that a classical system is an N x M function, taking a state of size O(N) and converting it into a (potentially smaller) state of size M.  So the state evolution matrix of a classical system is an N x M matrix.

Quantum systems, on the other hand, are reversible.  This means that information is not lost in a quantum system; it is just transformed.  So the state evolution matrix of a quantum system is an N x N matrix, and moreover a unitary matrix that preserves magnitude (though not distribution).  Hence quantum computations are fundamentally affected by interference patterns; they gave an example of how a quantum random walk winds up with a very different distribution from a classical random walk.

It seems that the problems quantum computers are best at involve symmetries.  Since a quantum computer is a very efficient way of transforming a configuration, it makes intuitive sense that it would therefore be a fast way to determine symmetry.  Factoring large prime numbers is an example of finding such a symmetry (corresponding to the factorization itself).

It’s a fascinating article and explains all this much better than I am.  Reversible computing is one of those Holy Grails that I truly hope arrives some year in a fashion I am capable of programming.  It could be the real answer to energy issues in computing.  On the other hand, it could cost more energy to create a quantum system with the right unitary matrix in the first place than you get back when reversibly running it.  I very much hope we’ll find out in the next two or three decades.

Written by robjellinghaus

2010/06/29 at 10:17

Posted in Uncategorized