robjsoftware.info

August was busy alright!

Despite all the business in my last post about SharpDX etc., I discovered that XNA 4.0 actually supports multiple windows quite straightforwardly, so being a lazy programmer I went with the easy route.  That version of the code is uploaded on CodePlex now.  I’m really happy with how it turned out!

Latency is Evil, Latency is Death

In my last post I also mentioned I met with some talented local loopers.  I didn’t mention that while demoing, a couple of them felt that Holofunk was just… too… laggy.  I was shocked by this as I’d been working for a long time to cut latency and I could only notice a subtle bit of it.  But they insisted it was no good.  So finally I thought to check the ASIO4ALL buffer size.  This was set to 512 samples, which at 48Khz (my current sampling rate) is just over 1/100 of a second.  That’s damn short!  But I shortened it yet further to 192 samples, which is 7/1000 of a second shorter.  And suddenly they loved it.  It felt seamless and right.   And I realized they weren’t kidding — 7/1000 of a second really is very audible!

It turns out that this is well known.  Looping technology is a sonic mirror, and if it doesn’t line up perfectly, it really throws you off.

All of this made such a big impression on me that I made a real point of it at the Microsoft demo I gave in August, which you can now watch.

That went well.  I won a runner-up prize in the “So Fun” category.  My use of a Wiimote kind of disqualified me, since it was a Kinect-for-Windows-sponsored contest and all.  I consider that fair; the applause was reward enough!

Then I did another gig at my sister’s wedding in Boston.  Unfortunately I ran into a bizarre situation I’d never encountered before:  my microphone sounded fine, but the Holofunk loops weren’t playing at all.  This despite the fact that there was only one stereo pair going into the DJ’s mixer!  I could not understand how this could be happening, and it almost blew my whole show in front of 150 people (no pressure!), but finally the DJ yanked one of the wires off and suddenly it worked.  Some bizarre kind of phase problem?  I hadn’t caught it during sound check because I just checked the microphone.  Lesson learned: SOUND CHECK EVERYTHING!  Still, it wasn’t a total disaster and many people told me they enjoyed it, so all’s well that ends well.

Plus, I took it to Cape Cod for our post-wedding family vacation, and my sister (a trained Bulgarian folk singer) turned out to be awesome at it.  She was very impressed and wants to play with it more.  Sooner or later I’ll need to package it for easier distribution.

Sound Weirdness Mega-Party

Then, Tim Thompson of Kinect Space Frame fame came into town for the Decibel Festival.  He very graciously wrote me and asked if we could meet and maybe do some kind of event.  He then contacted a local makerspace, Jigsaw Renaissance, and they got enthusiastically on board.  We wound up hanging out for a few hours, making many weird sounds, and having many interesting brainstorms with plenty of attendees.  It was all kinds of festive!

One highlight was meeting Tarik Barri, who had some cool looping video of various animated headshot clips of himself.  It was exactly the kind of thing I have in mind for Holofunk at some point — clipping out live video of the performer’s head, and looping that in place of the little circles, basically making a live version of Beardyman’s Monkey Jazz piece.  Check out Tarik’s animated face fun (slightly NSFW), and this 3D sound-space, which is also very inspirational for Holofunk’s head-mounted future at some point.

What was especially neat about this evening was that I felt, for the first time, that Holofunk can hold its own in a roomful of weird hacker electronic music projects.  It might not be ready for prime time, but it’s definitely ready for backstage with the big dogs!  Next year I’ll be working to get it into the Decibel Festival in some manner, for sure.

Hiatus, Interrupted

As far as actual coding goes, I’ve been in a low-hacking mode since the end of August, focusing instead on various other types of software, namely Diablo III, Borderlands 2, and XCOM 🙂  Fall is the biggest gaming season of the year and I’ve been indulging.  It’s been great.  But the worm is turning and it’s time to get back to Holofunk.

After adding the dual-monitor stuff, which turned out so well, I’m pretty clear that the next great feature is two-player support.  This is technically possible with just the hardware I have now — I can already connect two mikes to my USB audio interface, two Wiimotes via Bluetooth, and Kinect can do two-person skeletal tracking.  All I have to do is to refactor the guts of the code to support two of everything.  Right now I think I am going to let both players “step on each other” — e.g. each person can move and touch anywhere on the screen.  This will introduce some weird boundary cases, but should make for more entertaining play, if you can muck with the sounds the other person made.

The biggie after that is to add VST plugin support, particularly for the Turnado plugin from SugarBytes.  The reason that one is so important (despite its not inconsiderable cost) is that my inspiration Beardyman uses it in his new iPad-based software performance setup.  Bang, that’s the only recommendation I care about in the whooooole world.  Have a listen at this and imagine it in Holofunk.  Hell yes.

Of course, this is going to kick the complexity of the whole thing to another level altogether.  The main conceptual problem I’ve had for a long time is simply what the interface should be.  On the one hand I want it to be very simple and approachable; on the other hand, I want to be able to do ridiculously layered compound effects.  And, as with all Holofunk features, there has to be a smooth ramp from the simple to the sophisticated.

Here’s what I’m thinking:

• There’s an “effects mode” you can enter.
• When in “effects mode”, your hands aren’t selection cursors anymore; they’re “knobs.”  Waving the knobs around with the Wiimote changes parameters.  (Up/down = one parameter; left/right = another; forward/back = another.)  This would give six axes of parameter control just with your two hands, which is enough for starters (though really your feet will have to get in on the fun at some point…).
• There needs to be some menu interface for assigning parameters to axes.  In other words, you should be able to click somewhere and select left/right for pan and up/down for LFO frequency, or whatever.
• Then you should be able to set all your knob parameters as presets on the Wiimote D-pad, so you push up/down/left/right and instantly get a set of six parameters mapped to your two hands.
• THEN, holding down the A button activates your effects and lets you immediately mutate the sounds that you were just pointing at.
• THEN, holding down the A button and the trigger records your parameters as a loop!

So imagine an interaction like this:

1. Squeeze the trigger, laugh a little “Ho ho ho!”, let go.  Now you have a loop going “Ho ho ho!  Ho ho ho!  Ho ho ho!” forever.
2. Push the D-pad to pick the “pan = left/right” preset.
3. Hold the A button over your “Ho ho ho!” loop, and wave your hand left and right.  Now Santa Claus is jumping from the left channel to the right channel and back.  When you stop waving your arm, Santa settles down.
4. Now hold the A button and squeeze the trigger while you do a full circle with your arm, and then release it.  Santa Claus will now be looping from one speaker to the other.  Basically you recorded an animated envelope of the side-to-side panning.

The A button becomes the “apply effect” button, and the trigger retains its “record loop” behavior… you’re just combining them into a single gesture.  There still needs to be some way to affect the microphone itself (rather than just the selected loops) — I still need to figure that part out a bit better.  Maybe hold A while initially holding the mike and remote close together….

Basically, animated parameters needn’t have the same duration as the loops they apply to, and it should be possible to apply multiple animated parameters to a single loop or set of loops.  This should rapidly compose into brain-meltingly bizarre configurations of sound.  It might also make sense to add some kind of visual feedback for various parameters (e.g. use half-circles for fully panned sounds, etc.).  As with all Holofunk UI ideas, I don’t really know if this will work, but I do think it’s implementable and conceptually reasonably solid.

I also want to add scratching/sampling effects but I have not yet figured out how I want the interface to work, so I’m leaving that on the subconscious back burner until I get some kind of inspiration.  Just getting multiple animated, looped effects working will be plenty of amazeballs on its own!

So, Stay Tuned

It’s pretty mind-blowing that it was only a bit over a year ago that I was in Vancouver demoing all this to Beardyman.  It’s been a fantastic project so far and I expect it to become rapidly more so, as I get more features in and as I start collecting local technopeeps to play with it.  One of the other local loopers I demoed it to in June was Voidnote (and why did I not check out his Soundcloud before now?).  He actually wanted me to add guitar control to Holofunk, so he could use his guitar neck as a cursor, and use pedals in place of Wiimote buttons!  This is of course an awesome idea — how cool would it be to have a guitar/microphone two-player Holofunk jam?!  That’s on the radar as well.

My ultimate goal is to do a Holofunk gig at the annual Friends & Family rave campout.  I spent my twenties raving with that wacky bunch of Bay Area freaks, and I want to return as senior alumnus bearing live techno insane performance gifties.  That’s nine months away, and some of the hairiest features ever between now and then.  But if I can get two-player support by end of November, and animated Turnado support by end of February or March, and a couple more effects and maybe some video by June… then I’ll be ready!!!

Thanks again to everyone who’s enjoyed this project so far.  I’ve learned that one of my deepest satisfactions in life is working on a single project for years.  I’m loving raising my kids for that reason; my job remains excellent after four and a half years with no end in sight; and Holofunk is just getting rolling after 14 months.  Let’s see what the next 14 months bring!