I’ve been doing a lot of things lately, but one of them has been a prototype for what is basically an interactive music video. I’ve been collaborating with Paul Gratton and his brother Scott Gratton (of Finn Miles) through our collective Napkin Sketch, and this will be for their song Branches. The player would perform the piece to match up with the music in an appealing and expressive way. In this way, the experience is like playing a “visual instrument” of sorts. I think it’d be fun to use a Wii remote to do this, and once I get a Unity Pro license, that’ll be really easy to take input from. Until then, the interaction will be done with a gamepad.

It has been a big struggle trying to get an acceptable visual style going. I wanted to progress from the pretty rough interactive sermon prototype. I decided to go 2.5D, and by that I mean planes/sprites in 3D space. I’ve also been learning to write my own shaders a little, though at this point I’m mostly hacking ones I find on the Unity forums.

This screenshot was taken in the editor and shows grass dynamically moving away from a cube. It’s based on a script by metervara who kindly provided it to the Unity community. I’ve shared my version, which ports his shader to Unity 2.5 and adds the ability to grow the bending effect radius when you press a key/button. The intention is to allow the player to use that ability as one means of expression for a visual instrument.

The original context I imagined this being used in is during worship at a church. So the band would be playing musical instruments and people could play visual instruments as a complement. In fact, the first usage of this prototype will be at our church. The deadline is July 12th, which I will probably (hopefully!) make. Of course, this could be used in any live music setting, and in fact the three of us are planning to do that for a future project.

Oh! To continue the visual instrument analogy, I was thinking that the Wii remote would basically be like a tambourine in that the primary action would be percussive. This led me to think about other parallels, like a melodic device. And now I have dreams of circuit bending some sort of guitar thing to use for this idea, hopefully with the help of someone more engineering savvy. Just think how sweet it would be to have a circuit bent Guitar Hero guitar that played music and visuals!

Browse Timeline

• « Interactive Sermon Tool Prototype
• » Branches Interactive Music Video Prototype

Comments ( 1 Comment )

Add a Comment