As I explained in nauseating depth on my thesis page, I got a really weird Computer Science degree. I wasn't studying the sort of things people typically associate with Computer Science (inasmuch as people have any idea what Computer Science involves.) I certainly did my fair share of programming, but it was always a means to an end. As it is in the real world (i.e. not academia), the programming was invisible in the end product. So as a grad student, I got to do neat things with computers for a few years. (Like, four. It probably should have taken no more than two, but I was in no hurry.)
My interest in music would have been readily apparent to anyone within a few feet of me at any given time. Even a person who was blind and deaf would have been perturbed by the rippling waves of bass from the speakers strapped to my head. That's probably relevant.
I can't remember exactly how the ball got rolling and who pushed it, but my advisor had some vague association with this dude at the School of Music who was the musical director for of an improvisational music ensemble. Fortunately, someone had the good sense to bridge the gap between what we were doing in the HCI Lab (mainly, creating new ways for people to do exciting stuff with computers) and what they were doing in the more experimental factions of the School of Music (mainly, creating new ways for people to do exciting stuff with sound).
When I was first developing an interest in human-computer interaction, tabletop computers were all the rage. No one really had any practical applications for the things, but there were all sorts of sexy impractical things to do with them, so HCI research was very tabletoppy for a few years there. For context, this was around when Björk took the reactable out on tour.
So we built a table out of particle board and glue. No metal, because that would interfere with our electromagnetic sensor equipment that we used to detect where four pen-shaped wands were positioned. We also had to write the software to translate the position of those wands into touches on the table, which was just a dumb piece of wood. (I had great help; thanks, Ed Mak and Dave McMullen, for your significant contributions.) Then, we had to get creative.
The big observation that kicked things off was that most instruments are designed to be played by a single person at any given time. Tabletops, on the other hand, if they're good for anything at all, it's supporting teamwork and collaboration. So I conceived of three collaborative musical instruments.
The first instrument was a glorified drum machine, but it ran around the perimeter of the table. By virtue of how things were positioned, it'd be hard for one person to reach all of the settings; in order to program a complete beat, the group would have to work together. Each of the four people participating would be responsible for programming one quarter of the beat. We never made that one.
The second one used a metaphor of throwing ripples in water to make sound. Players would touch the screen which would cause ripples to be sent out from the point on the table that was touched, and there would be sound objects placed on the virtual surface that would play sound when struck by a ripple. We never made that one, either.
The third idea was to have players paint lines on the virtual table surface which represented walls, then use gestures to throw balls around the play area. When a ball strikes a wall, it produces a sound. The color of the wall determines the type of sound to be played, and the type of ball that hits the wall determines the quality of the sound. The speed of the ball controls the volume (velocity) of the sound.
We made that one.
I could never think of a genuinely good name for that instrument, and neither could anyone else. It had walls, and it had balls, so we called it WallBalls. The name stuck. I still haven't thought of a better name. I consider this to be a personal failure of the highest order, but that's neither here nor there. Crappy name aside, the instrument itself is actually rather cool, and the moment when we debuted the instrument in front of a hundred plus curious creatives on campus is one of my all-time special memories. I'll never forget the waves of relief that poured over me when the first ball hit the first wall and a sound actually came out of the speakers. A blue screen of death would have really done a number on my credibility.
Somewhere along the way, I decided that my Computer Science thesis needed more Computer Science and less Music, so the role of WallBalls in the whole thing was rather minimal. Still, I managed to get a paper on WallBalls published and it was personally very satisfying to be responsible for such a cool creative thing, despite its utter impracticability. I guess that's what academia is all about, isn't it?