Initial Project Proposal

My mission is to design a Multi-user, Multi-touch environment that allows individuals to work together within a musical space. A Multi-user table designed for Multi-touch interaction will be the point of collaboration. A Natural User Interface will act as the visual feedback where users can directly manipulate what is projected on screen. Within the design, a musical framework will be programmed that will make use of the table’s Multi-Input (Touch) ability.

While Multi-touch technologies have existed since the 1980s, not until recently have computer processors and graphical processing units been efficient enough to allow full implementation of Multi-touch environments. The recent developments of Apple’s I-phone, Microsoft’s Surface Computer, and JazzMutant’s Dexter are examples of the increasing use and popularity of Multi-touch devices. Of the three mentioned mainstream devices, only Microsoft’s Surface Computer can truly be thought of as both a Multi-user and Multi-touch device due to its larger scale. Although the technology is available, software applications that utilize Multi-touch input is still limited.

Musical application is a perfect environment that will benefit from Multi-user, Multi-touch interaction. Abolishing the limits of single point interaction will open up possibilities in the musical computing world. Examples of such possibilities include virtual Multi-touch instruments (pianos, drum sets, mallets, etc.), direct physical manipulation of virtual sound, and real-time Multi-user musical composition. The previous mentioned JazzMutant’s Dexter is a commercial example of a musical Multi-touch device. Unlike what I am proposing, the Dexter acts more as a virtual audio-midi controller rather than a single source of interaction.

Through the developmental process, I will be exploring both the hardware and software side of Multi-touch computing. On the hardware end, various materials (cameras, projection screens, etc.) will be tested and documented. On the software end, both low level (blog/finger tracking) and high level musical software development will be in the Open-Source domain as a part of the online forum NUIGroup (a Natural User Interface Group established to create a forum for Multi-touch device techniques).

To summarize, the final product produced will resemble a table with screen on top and hardware inside. The device will allow Multi-user and Multi-touch interaction with a focus around musical applications.