Framing the discussion

Problem statement

It is anticipated that the true wireless headphones market will exceed 200M units and will generate revenues in excess of $34 billion dollars by 2024. When considered as part of a larger hearables market which includes personal hearing devices and glasses, hearables are disrupting a traditionally high-volume, low-margin market with devices demanding higher ASPs and offering more convenience, capability and use. Hearable devices are benefiting from advances in low power digital signal processing, Bluetooth technology, micro-speaker design and an increasing number of sensors that can expand the capabilities of today’s devices.  In order to benefit from the opportunity, the group focused on identifying the state of the art with headphones today, reviewed some of the problems associated with today’s wireless headphones and then posited on anticipated capabilities, modalities and use-cases for a next generation of hearable device.

Outcome

It’s clear that in the hearable market today, battery life (power) in a hearable form-factor is the single biggest limiting factor when considering expanded functionality in these devices.  In order to identify new areas and concepts for hearable usage (such as environmental awareness, safety, health, etc.) the team assumed that basic power requirements would be solved, and then attempted to get to a point where a common platform architecture for a next generation of hearable could be defined.  The team did not succeed in this goal, but did provide a review of existing functionality, identified problem areas with today’s devices and made recommendations for future hearables.

What do today’s headphones offer beyond basic playback and communication?

The group took a 3-prong approach to review the capabilities of today’s true wireless hearable devices.  First, the team identified common use cases offered by today’s headphones.  It then looked at problem areas to be solved and then finally raised some key questions and recommendations for consideration when defining a next generation of hearable. 

State-of-the-art

Looking at today’s true wireless headphone market, the team first identified some of the common use cases and segmented the use cases into baseline requirements (i.e. expected functionality in devices today), state-of-the-art to describe use cases that are either supported today with high-end hearables, or features that have been announced and anticipated to ship within the next year.  Finally, potential future requirements highlights some of the ideas that were captured, together with some hurdles that would need to be overcome in order to successfully implement the feature.

Issues with current headphones

As a next step, the team used both primary and secondary research to consolidate issues with current headphones.  Primary research was based on interviews with the team (along with several other subject matter experts during the course of BBQ) and secondary research mainly consisted of an internet review. The team identified the following problem areas with today’s headphones and where possible, attempted to define a fix together with identifying a functional solution.  The table below summarizes this discussion.

Building the next generation of hearable

Some design considerations

During the brainstorming session and review of the state of the art for true wireless headphones, the team gathered a number of key questions to be addressed when considering the functionality of a next generation hearable. These questions are captured in the table below and can be considered by future groups. Note that the team considered mechanical fit and aesthetic design outside the scope of the discussion.

Proposed Functionality

As a straw-man for a next generation true wireless hearable device, the team came up with the following functional design goals.

The Device should be able to determine the user scenario and adjust the device to support the required use case.

• for example, you are on a phone call and the microphones are focusing on your voice.
• once you are off the call, the microphones focus on your environment and amplify/reduce the volume and ambient noise as necessary
• when in an external conversation, the microphones will focus on the person(s) you are talking to
• This will help to avoid social awkwardness by wearing headphones
• people will become used to everyone wearing headphones
• When riding a bike or walking down the street, the headphones will sense dangers and alert the wearer.

The desire is to remove the need for multiple headphones

Anecdotally, each member of the team has more than two pairs of headphones.  There are headphones that one uses at the gym, headphones for travel, headphones used primarily for communication.  While the industry benefits from this segmentation, the group consensus was that it would be better to create a single pair of headphones that could support multiple use-cases.  It was also identified that these devices should be easily discoverable and find-able.  Loss of true wireless headphones is perceived as a growing issue.

The headphones will support multiple input sources and intelligently switch between them.

While investigating the use of hearable devices, connectivity came up time-and-again as an issue for current headphones.  Future requirements should enable a headphone to intelligently switch from a personal device to an external devices (like the airplane infotainment system or advertisement billboard) based on user-defined preferences.

The next generation of headphones will be an integral part of a whole-body-network

As stated above, the team believes that the next generation of smart headphones will become part of a whole-body network and will be able to act seamlessly with other personal devices (phone, glasses, watch etc.) to apply the appropriate processing in the right device (with seamless handover) to drive optimal power management, and also to drive a consumer centric user experience.  This will pose interoperability challenges for the industry and we believe will lead to further vertical integration of solutions (as can be evidenced by Apple, Samsung, Huawei etc.).

Quality of Life

The team feel that while these devices will, at their core, continue to be primarily entertainment consumption and communication devices, there is a large opportunity to improve a consumers quality of life, particularly for the sensory impaired.   The ability to embed intuitive environmental and context awareness into hearable devices can lead to opportunities to improve life quality that other personal devices cannot achieve.  Hearing is a core sense, and a hearable is ideally positioned to intelligently enhance or augment that sense for the consumers benefit.

Personal

We believe that we will see a continuing trend towards personalization of the audio experience (personalized hrtf models, personalized EQ for hearing loss, privacy filters for audio etc.) and that in addition to form and fit, personalized functionality will play an increasingly important role as the devices get smarter.

Observations and Conclusions

The team believes that hearables will continue to be highly relevant personal devices.  Purchase decisions will be made based on form, function and fashion.  Consumers will continue to desire the fundamental modality of the headphones; i.e. the ability to escape the environment in order to focus on the content, and will want that experience to continue to grow, both in quality of audio, ease of connection and increasing battery life. 

The team also believes that consumers will gravitate towards a new set of use cases outlined above focusing on health, wellness, improved contextual awareness and augmented audio capture but only if the proposition is sufficiently differentiated (i.e. can’t be better satisfied by another personal device),
In terms of the underlying compute platform required to enable new use cases, it is going to be important that the devices perceive the appropriate user context using a combination of sensors to determine (a) where the user is and (b) what they are doing.  This will likely require an edge-based machine learning architecture which will have an impact on both the compute and memory architecture.  Additionally, if these devices continue to use more sensors to gather contextual and environmental inputs, it is likely that the number of required inputs (say 6 microphones, 2+ speaker drivers, potential camera inputs, accelerometer etc.) will determine the overall size of the package and not the underlying ML core.

References

[1] Wireless Headphones Market - Global Outlook and Forecast 2019-2024 https://www.marketresearch.com/Arizton-v4150/Wireless-Headphones-Global-Outlook-Forecast-12142037/

[2] Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors https://www.nature.com/articles/s41598-018-32063-4

[3] Affective Signal Processing (ASP): Unraveling the mystery of emotions
https://ris.utwente.nl/ws/portalfiles/portal/6035209/thesis_E_vd_Broek.pdf

[4] The Complete Guide to Hearable Technology in 2019
https://www.everydayhearing.com/hearing-technology/articles/hearables/

Appendix

In addition to focusing on the basic hearing/talking functionality that is a core requirement for today’s headphone, the team created the following mind-map identifying some of the different modalities and use-cases that could be offered by a next generation hearable.

section 9

next section

select a section:
1. Introduction
2. Workgroup Reports Overview
3. #KidsHearingMatters: Designing a Safer In-Ear Listening Experience (for Kids of All Ages)
4. A generalized haptic format standard leveraging existing MIDI infrastructure
5. Using Machine Learning to improve the experience in consumer audio devices
6. Audio Turing Test
7. The Future of In-Ear Voice Assistants (Assistance)
8. Developing a set of audio requirements for the next evolution of Bluetooth
9. Headphones Unlimited: Current Problems and Future Opportunities
10. Is this the real world? Or is this Fantasy?
11. Schedule & Sponsors