Brief statement of the problem(s) on which the group worked:

In the future the full acoustic spectrum is required for new use cases that require non-traditional sound. Some examples of this include:

• Inference (acoustic awareness)
• Transmission of data utilizing ultrasonic communication
• Imaging the room using ultrasonics
• Simultaneous use of traditional audio and non traditional sounds

Developments in transducers have unlocked new use cases beyond traditional acoustics.  These unlocked feature sets struggle to be implemented seamlessly.

How is the data from these multiple components formatted and packetized:

• How is it transmitted across the system?
• Where is it processed?
• How much is in the cloud?

Therefore, there is a need for aggregation of data from multiple components, some of which are not audio specific, while at the same time some traditional audio components are being used for non-audio applications. 

A brief statement of the group’s solutions to those problems:

It was decided that it was better to “turn things upside down” by identifying problems to highlight to the industry the areas that need to be addressed.

The following are some meaningful terms that we used in this report and to allow communication in a consistent and understandable fashion:

• Audio: “traditional audio”... spoken communications, music, etc
• Sounds: full spectrum of acoustic information (not including “traditional audio” as defined above)

The focus of this report is on the non traditional sounds and processing to ensure that it does not negatively impact traditional audio.

Some recommendations:

• Cloud based application vendors (Alexa, Cortana) to support non traditional sound ecosystem.
• Cloud providers need to participate in standards activity.
• Industry (transducer suppliers, SoC, Apps, Cloud) should adopt new interfaces to address the extra data collected from the non traditional sounds.
• More collaboration is necessary:
  • Common feature sets among like technologies
  • Between cloud application providers

Expanded problem statement:

The full acoustic spectrum is required for new use cases that require non-traditional sound. There is an insular market dynamic which is prohibiting innovation and adoption of enhanced usage of existing transducer and interface capabilities. Every stage of the existing architecture faces problems addressing this new paradigm, some of which are identified below:

Transducer interfaces (analog, PDM, I2S):

• No local compute
• No proper power controls
• Limited discoverability
• Limited number of function specific streams
• No gain control or AGC
• Audio disruption changing power states
• No info can be gathered in the interface
• Multiple gain structures are not supported such as those needed to support non traditional sounds

SoC, Application, Cloud interfaces:

• Insufficient communication between compute layers
• Difficult to add value if not standardized
• Insufficient protection and security around the communication, extremely vulnerable to a “man in the middle” attack, no way of detecting
• Need to transport metadata and classification data

Shared usages:

• For audio content capture or transmision the user has control on how it is captured and produced (i.e. volume control in music playback). The non traditional sound capture the user should not have that control (i.e. gain control in ultrasonic room sensing).
• Aliasing of high frequency content into the spectrum degrades the traditional audio
• High amplitude high frequency content can clip the signal
• Doppler effect distortion
• Dynamically change all operating points (gain, power, fidelity) without audible artifacts

Multi-device collaboration across the signal chain hierarchy (transducer, SoC, app):

• Communication required at multiple stages of the signal chain across multiple devices
• Multi channel/device clusters:
  • Microphone arrays
  • Playback arrays
  • Bandwidth and latency issues are inevitable as the clusters get bigger and bigger
• Traditional processing is focused only on audible spectrum and non traditional sounds are discarded
• Pollution from dissimilar ecosystems
• Environmental sensing and mapping
• Non traditional audio privacy and security concerns have not been resolved

Processing:

• Substantially higher processing burden by including the full spectrum
• High need for the fixed function processing vs general purpose computing to stay at the same power consumption level
• Transporting of non traditional sounds vs audio is not being done
• Processing of non traditional sounds will require neural network processing at the application processor or DSP level

Architectural knowledge about the non traditional sound processing:

• Different layers within the architectural hierarchy are unaware of the processing made in other layers
• This is a problem today in traditional audio where there is no awareness of what processing is being done. The next architecture will have an even greater challenge when there is more processing being done to support non traditional audio.
• Discovery of lower level compute capability

Closed ecosystems that only support traditional audio:

• Assistants trying to control the entire value chain
• Do not facilitate non traditional sounds (band limiting)
• Zero interoperability between ecosystems
• Local or device level inferencing not supported by ecosystems
• Stifles innovation

Expanded solution description:

Other reference material:

SoundWire, where are you? https://www.linkedin.com/pulse/soundwire-where-you-peter-cooney/

Audio Sensor Opportunities: Market Requirements and Technology Challenges for the next Decade https://www.projectbarbq.com/reports/bbq16/bbq16r4.htm

SILENSE (ultra)Sound Interfaces and Low Energy iNtegrated SEnsors, https://silense.eu/project

section 9

next section

select a section:
1. Introduction
2. Workgroup Reports Overview
3. An Exploration of Machine Learning and the use cases where it might provide the most benefit for Audio Synthesis
4. Benchmarking methodology for a multi-voice assistant enabled future
5. Problems and Solutions for Audio in Augmented Reality Headsets
6. A World Without 3.5mm: Transport Features, Guidelines, and Opportunities
7. Pork Rinds: Challenges with the present hearable model
8. Taking the "virtual" out of virtual audio
9. Impact of non-traditional sound: mic used for ultrasonic, etc. Everything is broken!
10. Schedule & Sponsors