Problem Statement

Standing on the shoulders of pork

The emerging hearable market poses both an opportunity and a threat to consumers wishing to compensate for hearing defects.  This paper expands on a previous BBQ Workgroup project, Bacon for your ears: Designing a Musical hearing enhancer and outlines new problems, opportunities and potential next steps that we can take, as an audio industry, to ensure that consumers’ hearing and well-being are managed appropriately.

Background

In August 2017, the US passed the Food and Drug Administration Reauthorization Act of 2017, legislation that includes the Over the Counter Hearing Aid Act designed to provide greater public accessibility and affordability with over-the-counter (OTC)  hearing aids.  The act is designed to enable adults with perceived mild-to-moderate hearing loss to access OTC hearing aids without being seen by a hearing care professional. It comes on the heels of the elimination of the “physician waiver” system which had required consumers first to seek a physician for a medical evaluation or sign a waiver prior to obtaining a hearing aid.

This change in law has accelerated growth in a still-nascent consumer smart hearable device market, which extends the features of headphones to support augmenting consumers hearing based on individually defined criteria.  The hearable market is an exciting new market with high projected growth and contains a range of “smart” devices that offer everything from the auditory equivalent of “reader glasses” to sophisticated hearing aids with expanded headphone functionality.  There are already a range of devices available on the market from established consumer electronic companies, e.g. the (Bose Hearphones), medical device companies, e.g.  (Starkey Halo 2) and startups e.g. (Bragi Dash Pro).  Broadly they offer either augmentation or measurement based hearing correction.

Opportunities and risks

The problem an unregulated audio-correction market presents is that it creates a potential for low-quality CE headphone-like devices to thrive and these devices may augment your hearing by applying techniques that are at best ineffective, and in the worst case deteriorate your hearing when worn over a long period of time. 

The targeted device that raises concerns is the low-cost “we-can-make-your-hearing-better” category of devices that may be available in supermarkets or pharmacies, much like a pair of reader glasses are available today.  The difference is that while wearing “reader glasses” may not be destructive when worn over a long period of time, it is posited by this team that prolonged exposure to hearables that contain poorly specified audio compensation algorithms may affect long term hearing.

The challenge for the Consumer Electronic industry is that a unified standard or interface for non-prescription consumer hearing-compensation device does not yet exist in order to ingest audiologist authored prescription data.  Think of the equivalent of going to an optician to get a new pair or prescription glasses.  An ophthalmologist measures your eyes and then creates a compensation specification.  This specification (prescription) can then be taken to any authorized optician to purchase a pair of glasses that meet the consumers style, budget and functional requirements.  This paper proposes an equivalent for the emerging hearable market.

Since 2012, the capabilities of machine learning and artificial intelligence have rapidly advanced. Machine learning offers incredible opportunities for early detection of hearing problems, and for innovative processing techniques that move away from the algorithmic. However, there is a lack of available data on hearing capability, particularly for the years before a hearing impairment presents itself. The collection and development of large, anonymized, standardised set of hearing data is a related challenge that we address in this report.

Proposed Solutions

Summary

We propose a three-part solution for these issues.

1. A standard format for capturing and sharing a hearing prescription
2. A logo certification program to certify proper support for that prescription and minimum requirements for hearing safety
3. The collection of a large-scale dataset of these prescriptions to enable machine learning applications in diagnosis and processing

Standard for capturing a hearing prescription

Our brief literature review has identified that presently, the most widely used audiometry tests restrict themselves to plotting the level of the users threshold of hearing against frequency in octave bands from 250Hz to 8kHz for each ear.  Here we use the word “plotting” literally to mean making marks on a piece of paper.  After extensive research (Googling for half a day) we have been unable to find any electronic interchange format (common or even proprietary) for this data.

Requirements for an open interchange format for an audio prescription

1. At a minimum an interchange format will need to include the data described above.
2. We believe other measurements will also be required, such as plotting changes in the threshold of pain in similar frequency bands.
3. We suspect that a method of electronically signing such a format will need to be adopted such that both the audiologist creating a measurement can the subject of the test can be assured that data is unchanged.
4. We’re unsure of the maximum amount of data that will be needed to characterize a users hearing but we hope that this can be restricted to something around 2kB.  If this is possible the interchange format could be encoded onto a QR code allowing the subject to carry with them a machine-readable paper manifestation of their hearing.
5. As with any other format potentially aimed at standardization we intend that the format include no proprietary intellectual property.
6. It is intended that the format be open but the data be private and owned by the user in order to meet global data compliance regulations.

Logo certification program

Summary of proposal

We propose a collaboration of the BBQ brain, audiologists and the CE industry to align on an opt-in logo certification program for hearables. The logo would be a consumer-facing mark of safety and indicate proper support of the hearing prescription specified above.

Benefits for the consumer

The consumer benefits are clear: confidence that certified hearables will not risk damaging your hearing, will be simply and accurately configured to your hearing prescription, and will play nice with other devices. In addition, by enabling a quality assured hearing compensation program, the following consumer benefits are posited.

1. Stigma: Reduction in the stigma surrounding hearing loss
2. Convenience: Combining hearing compensation with traditional headphone use cases, we reduce the number of devices that a consumer needs to manage, charge and maintain
3. Choice: By adopting an open interchange format consumers are free to purchase devices that meet their branding, fashion and budget desires
4. Personalization: by facilitating a common language between between audiologists and CE manufacturers, it allows for further personalization of the hearing for specific use cases

Benefits and challenges for the industry

Having recently shaken off the shackles of restrictive and unworkable FDA regulation, the hearables industry is unlikely to be enthusiastic about a new certification program. It is imperative for the success of any such program that it be built by the industry itself, and be opt-in in nature.

The benefits for the industry are clear. By creating a logo that provides a defacto quality standard for the industry, each company is insulated from a future scandal of hearing loss that could collapse the whole market segment. 

By complying with a hearing prescription format, companies guarantee that users can begin using their devices immediately and over a longer term. Such a logo program should not stifle innovation in processing or in form factors.

Data set for machine learning

Summary of proposal

We propose to enable and promote the collection of a large-scale data set of anonymized hearing prescriptions to drive future machine learning applications in the hearables and hearing-aid industries.

Opportunities

The largest opportunities of such a data set tend to be unclear until they are discovered, which they never would be without the existence of said data. However, the two clearest applications are twofold: the potential detection (and thus prevention) of future hearing problems before they occur, and a transition to machine learning processing for hearing enhancement from today’s primarily algorithmic models.

Such a program of data collection could be driven by the logo program. By partnering with a medical university or large insurer, hearing tests conforming to our proposed prescription standard could be delivered as standard over many years as part of an annual checkup. In time, a transition to the inclusion of regular hearing tests in the same manner as vision tests or dental checkups could help to reduce the current stigma around hearing devices.

Challenges

The biggest challenge with any collection of medical data is privacy. There are also trends towards greater privacy and data protection across the wider technology landscape with legislation such as the EU’s landmark GDPR regulation which came into force in May 2018. Any data capture and storage must comply with these strict rules, and be held and managed accordingly.

The funding of the capture of this data would also be an issue to be addressed. This funding could be pooled from contributions of the logo program’s members. It is our contention that the benefits of a large, common data set outweigh the competitive advantages of a small, private one.

Future work (gen 2+)

We propose the following subject areas which future BBQers may choose to investigate as a follow up to this proposal:

• Inclusion of metadata in content offering custom tunings for different hearing profiles -- and investigation of whether there is a commercial case for this
• Usage of this data to drive speaker innovations for the hearing impaired
• Closed captioning of real-time speech in AR / MR / VR
• Parameterized voice description
• ASL to speech / speech to ASL

Further reading and research links

http://NIDCD.NIH.Gov
http://HearingAidKnow.com
http://HearingAid.com
http://Audiologyonline.com
http://HearingReview.com
http://BabyHearing.org
https://www.audiology.org/sites/default/files/publications/resources/20180130_AuD_Guide_OTC.pdf

section 7

next section

select a section:
1. Introduction
2. Workgroup Reports Overview
3. An Exploration of Machine Learning and the use cases where it might provide the most benefit for Audio Synthesis
4. Benchmarking methodology for a multi-voice assistant enabled future
5. Problems and Solutions for Audio in Augmented Reality Headsets
6. A World Without 3.5mm: Transport Features, Guidelines, and Opportunities
7. Pork Rinds: Challenges with the present hearable model
8. Taking the "virtual" out of virtual audio
9. Impact of non-traditional sound: mic used for ultrasonic, etc. Everything is broken!
10. Schedule & Sponsors