64,795 views 45 comments


AudioTouch is an interactive multi-touch interface for computer music exploration and collaboration. The interface features audio based applications that make use of multiple users and simultaneous touch inputs on a single multi-touch screen. A natural user interface where users can interact through gestures and movements, while directly manipulating musical objects, is a key aspect of the design; the goal is to be able to interact with the technology (specifically music based) in a natural way. Users can load, exit, and move musical applications using gestures as well as have a multiple applications operate within the same work environment. The design is built into a table that hides all the hardware necessary for tracking multiple inputs as well as displaying visual feedback on the touch surface.

video of the applications on my multitouch table


Currently, the AudioTouch OS consists of four main musical applications. MultiKey, the first application, is similar to a typical musical keyboard or piano. The application includes two polyphonic musical keyboards, one being a full octave, and the other being two octaves. Users can resize, zoom, and rotate a keyboard by using a two finger gesture.

Musical Squares

The second application, MusicalSquares, consists of color coded squares that bounce around within a bounding box. As the squares come in contact with each other, they collide, play a musical note (based on their color), and then bounce off each other in opposite directions. A user can press on a square and make a throwing gesture which causes the square to be thrown in the motioned direction.

Audioshape Seuqencer

Audioshape sequencer, is a third applications that allows users to create musical loops. Squares can be moved around and connected. When the play button is pressed on a square it plays through the connected squares in sequence.

Musical Wall Pong

Lastly, Musical Wong, is a multitouch game similar to pong/hockey. When the ball hits the walls, different sounds are played.

For information about the process of building the table and software, please check out the MTmini , my multitouch progress, and software sections of this site.

Did you enjoy this post?

Comment Below , Follow Me on Twitter , Follow Me on Facebook , or Network with Me on Linkedin

  • Keep up the great work! I linked the video to my Interactive Multimedia Technology blog.

  • cerupcat

    Thanks Lynn!

  • Looks sweet Seth!

  • crypto

    keep up the good work….this is amazing….

  • Hi Seth. Fantastic Work. I love the pong and the loop machines for more traditional making. THe throwing squares are look fantastic for ambient things.

    Has there been any pedagogy developed for any of the instruments?
    Nice to know that MIT doesn’t have all the good stuff.

  • taam

    help me no of the softwar is campatle with me computer for the miny one so help me!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  • timominous

    hey seth, check out the readers digest asia may 2008 version at page 13. .there some king of table there that is somewhat similar to yours used in a concert. .

  • cerupcat

    Hi timominous,

    Where can I see this? Is there a free source online?

    You’re probably talking about the ‘reactable’ which was ussed in concert with Bjork.

  • timominous

    yeah. .that was the one. .

  • Pingback: Audio Multi-Touchscreen()

  • Pingback: Sonic Residues » Blog Archive » AudioTouch « AudioTouch and more()

  • al peters

    Hello seth,
    i have not yet built my homemade multitouch controller but plan to shortly. I plan to use it for music. Your information looks very helpful. I was wondering if you had seen this application:
    what would be the software issues involved in making a touchscreen control a harmonic keyboard layout such as this (especially for controlling midi)? Im afraid im not massively technically literate ; (

  • cerupcat

    al peters: Yeah I have seen that video. It’s great! I was planning on making an application like that eventually (but without the need for physical markers). Well, we have software @ nuigroup.com that sends out OSC messages. OSC is like the new MIDI. So basically it tells you what x/y you pressed and you can use that however you want.

    You can find more information out over at nuigroup.com where we have a bunch people building hardware and writing applications just like what you posted.

  • Brilliant!

    Have you connected the music with numbers or words?
    Imagine hearing music attached to key words or numbers as you scan a document. Even better is the possibility of attaching custom music samples to words to echo Persona Behaviour.

    I’ll ask one of my technical friends to look at your software.

    For more on the Persona Behaviour concept go to http://www.scenario2.com or scenario2.wordpress.com.

    Cheers and good luck!

  • Pingback: “Let the Music take you!” Music meets touch to power Personae « Stories power Markets()