home  previous   next 
The Twelfth Annual Interactive Music Conference
brainstorming graphic

Group Report: Next Generation Hand/Glove Controller

Participants: A.K.A "Hand-Aid"

Keith McMillen, BEAM

Steve Turnidge, Ars Divina


This group wanted to explore designs for a non-intrusive UI extension to existing interfaces and instruments. This resulted in the description of The Hand-Aid – a hand shaped band-aid like muscle & motion detector, which adheres to the back of the hands.

Problem Statement:

Nostalgia for data gloves without the drawbacks that led to their extinction. Extract gestures from instrumentalists to use as additional control information. Older data gloves were heavy and hard to calibrate and generally excluded the wearer from using his hands for any other activity such as playing an instrument or operating a controller or computer.

Solution Description:

Since so many people have mastery over an interface or a musical instrument, we wanted a non-intrusive way to enhance control and expression during normal use of these existing interface methods.

In the future will have wireless disposable adhesive band-aid like sensors shaped to the back of your hand and can detect finger placement, motion, exertion and location via measurement of muscle movement (electromyogram). They will come sealed on a paper sheet (like a bandage) and you rip off the cover and the adhesive side is facing up. You place the back of your hands onto the conductive fabric and it adheres. The Hand-Aids are self power, wireless and are aware of each other.

Most of the technical requirements needed to actually advance this concept to reality are 5-10 years off. Several papers have been written on using RFID technology for actual location of missing inventory. I believe demand for this functionality for large scale problems will trickle down to more artistic uses such as the Hand-Aid.

Discussion with experts on extracting data from surface mounted EMG (electromyogram) sensors is not presently possible. The signal to noise produced by these sensors is very low and would require new data recovery techniques for low signal level extraction. Some work is done in this area for radio telescopy and may be adapted in the future.

Electrically useful fabrics are starting to become available. While pressure sensitive fabrics do exist fabrics sensitive to tension (stretching) are not yet available.

Enhancement Mode: The output can be directed to already known instruments. Examples: Guitar pick grip pressure mapped to wah-wah, Assigning  a given finger to a voice for piano (So the left hand pinky can always be a bass and the right hand thumb a Flute) ,   While you are mixing you could morph through EQ settings or add verb by additional pressure on the faders.

Stand alone mode: Classic data glove/ VR  behavior. Now tapping  on a table could actually be a drum controller. Air conducting could control tempo or voice volume. Control of traditional computer functions commands from gestures.

External Controllers: Using it as an external input to ancillary devices such Light-shows, input to other performers,  control of sequencers, loopers, Algorithmic composition engines.

Extra bonus feature: sign language to voice or text converter.

section 9

next section

select a section:
1. Introduction  2. Speakers  3. Executive Summary  
4. Th Unfnshd Smph... Fixing Broken PC Audio
5. iHear the Future
6. Overcoming Roadblocks in the Quest for Interactive Audio
7. Call for a Highly Distributed Metadata
8. Game Producer's Guide to Audio
9. Next Generation Hand/Glove Controller
10. The Computer as a Musical Instrument
11. Schedule & Sponsors