Karen Collins, University of Waterloo

Brief statement of the problem on which the group worked:

What’s going on with mobile audio applications?

Cellphones, netbooks, smart appliances, smart instruments… the capabilities of mobile devics are changing really quickly. Can we predict the audio aspects of “killer apps” in tomorrow’s mobile world, and the type of features we’ll need to create and support these apps?

Audio people and companies need an evolving understanding of where mobile audio is heading, and what opportunities are available for developing hardware and software. This workgroup report will assist with this particular problem.

A brief statement of the group’s solutions to these questions

We don’t propose a prescriptive answer to lay out the future of mobile audio. Instead we outline a framework for understanding its evolution, on three axes:

1. General technology trends (across hardware, software; economics and business models; social and environmental context). We focus future goals by comparing “utopian” against “dystopian” outcomes.
2. Stakeholders (users, hardware and software developers, service providers), and categories of application. We can focus on application areas using personas and usage scenarios.
3. A value judgment on the desirable qualities of an application, looking at key characteristics of popular applications. Brainstorming produces some new examples of “killer apps” as a discussion point.

1. Technology Trends

Background

We use the term “mobile” here to include the trend toward small portable devices for everyday ubiquitous use. All these mobile devices have digital – often wireless – interfaces, connecting them to the wider environment including nearby devices, Internet services and people. A mobile application’s value comes from connectedness as much as from mobility.

Sensors, Communications, and Environment

We anticipate increased growth in sensor technologies. There will be yet more sensors, and sensors in more devices. It is increasingly common for cellphones to have an array of sensors such as microphones, camera, GPS, accelerometers, compasses and gyros. We anticipate an increase in the quantity and also the diversity of sensors in mobile devices, to include humidity, temperature, proximity RFID, and biofeedback.

Some of the future applications we’d like to see rely on accurate location sensing beyond the resolution of GPS – relative location within a fraction of an inch, for example. It’s too early to tell whether such sensors will exist and become widespread.

The combination of sensors with Internet communications will create new capabilities in terms of the applications we will see, for instance, geolocation, geotagging, social location, and augmented reality applications. While these “augmented location” applications are being developed, they have (with a handful of exceptions) yet to have an audio-driven component, with a few exceptions:

• Audio notifications (beeps, warnings etc),
• Augmented-reality audio, which is still mostly an early research topic.

These applications will often span different interaction modes and situations. For example, a single “application” can span home, car and phone. We may want to carry the audio context – music, for example – across these without disruption (e.g. from MP3 player to car). We anticipate applications having modular components where each module might be specialized to a particular type of device or a particular usage situation.

Audio Interfaces

Loudspeakers are an ongoing difficulty with mobile products. Portable speakers remain expensive and a niche market, and yet it’s very hard to get adequate sound quality from built-in speakers. We hope that this situation may improve, so that a user may, for instance, project a presentation from their phone onto a wall, and have great audio to accompany that projection.

Most visions of the future seem to prominently feature visual displays everywhere, from wall-sized panels to tablets to flexible OLEDs wristbands. The “audio display” technologies, especially for portable devices, aren’t so easy to define. Perhaps the “earpod” (see Bar-B-Q 2007) is such a compelling proposition that we might see headphones worn almost permanently; cellphones have most of the necessary features already. We don’t see an equivalently flexible loudspeaker technology yet, despite the need and the increasing pressure for thinner portable devices. Mass-coupled (e.g. gel speakers) or bending-wave transducers might be able to meet these scenarios, and there’s lots of potential for wirelessly networked “audio display” infrastructure (loudspeaker arrays, etc.), but we think this will remain an important and difficult problem area.

Business models and Applications

In the iPhone application market it’s difficult to build sustained business models with a massive supply of cheap-to-free alternatives. As multiple mobile operating systems become common, developing across platforms will probably remain quite expensive. In-application purchases can help to build a sustainable business. We also see a trend where the end-user’s mobile applications are “free parking”: a subsidized part of a larger service/product offering.

In order to compare potential positive future with a potential negative direction for mobile audio, here are some utopian and dystopian forecasts of the future:

We anticipate that there will be a variety of industries behind growth in mobile application development, and who will benefit from the creation of standards, new technologies, and collaboration. In particular, we see the following industries as crucial to the future of mobile applications:

• Health (for personal health monitoring, fitness training and tracking, stress monitoring, mood, bio-awareness, mood adjustment by automatic music choice based on schedule, etc.).
• Smart appliances (for notifications, starting/stopping appliances from a distance, etc.)
• News and information services
• Media and entertainment, sports; gambling, games, television, movies, etc.
• Creative tools and collaborative music.
• Social connectivity
• Education and learning

Music (and other audio) is often consumed as a background for other activities, and we expect that trend to increase. The time when users sat and “just” listened to music seems to have passed; audio is often secondary to video, and this seems to be even more the case in mobile applications. However, the limitations of small screens and the unobtrusive nature of audio are in its favor, even for active entertainment that’s not primarily driven by its audio content.
Moreover, consumers seem content to consume music in MP3 format at diminished fidelity. We would hope that, as the technology improves and cheapens, that improvements in sound fidelity will drive users to recognize differences and demand better.

3. What makes an application “good” or “bad”

We surveyed a large range of audio-related applications and features – especially portable or cellphone-based – ranging widely across games, musical instruments, players, teleconferencing systems, alarm clocks and notifiers, home appliances, pedometers, voice recorders, ebook readers and audiobooks, calendars, and more.

From this survey we looked at the successful and unsuccessful applications, using some subjective criteria to identify desirable and undesirable characteristics:

From this table it’s easy to see a correlation between a good application and our experiences of “good gameplay”. Lessons learned from game development – structuring a deep interaction with an extended experience – will be invaluable in the design process for future mobile applications.

section 3

next section

select a section:
1. Introduction
2. Executive Summary
3. Hear, There, and Everywhere
4. Dick and Jane Tracy plug into the Matrix
5. Mobile Infra-Structure
6. Re-imagining Operating System/Hardware Services for Applications
7. Schedule & Sponsors