David Rowe, Neversoft

Problem

Current spatial audio authoring and delivery systems are not always aware of the rendering environment, so the content is not always rendered in a way that preserves the artistic intent.

Goal

A scalable audio spatialization system that preserves the original artistic intent. This system should support authoring, transmission and rendering of audio that adapts to the environment, the listener(s), and the hardware.

Prior Art

• VBAP
• Ambisonics
• iXMF
• 3DAA
• AudioStage

Honey File Format

Authoring Process

Rendering Process

Server to Renderer

Use Cases

Music video authoring “Linear Larry”

2011

• Stems/5.1/stereo-matrix/mono/headphones

2016

• Stems & Metadata à *.hon file
• YouTube queries environment and renders as appropriate
• Small speaker, full-range speaker, number of speakers, headphone, dynamic compression

• Environments, objects, distance, vector info
• Look at iXMF, add querying
• iXMF Hardware Platform
• Audio in app (trigger)
• Interactive (eg, voice, game variables)

Voice Communications “Mobile Molly”

Conference call with aggregated phones (M phones, N users)

• Server aggregates input, creates .hon stream
• Each device renders appropriately
• Language translation support

Cinema “Zack Zimmer”

• Awareness of audience
• 3D movies (authoring)
• Earbuds & subwoofers
• Transmit sound effects like whispery voices to individual phones in the audience

Musician “Performance Pete”

• In ear monitors for stage
• Stream in .HON format (local and remote)
• Fans can isolate instruments, e.g. guitar
• No bad seats
• Stage vs. audience perspective

Tablet educational app “Sally Student”

• Feel like you are in the lecture hall
• Distance education – send voice back and forth
• Language translation

Conclusion

 section 4

next section

select a section:
1. Introduction
2. Workgroup Reports Overview
3. What's Next for HD Audio?
4. Making Spatialization Work Within Constraints of New Form Factors
5. Creating Mobile Audio Architectures that Support Flexibility and Differentiation
6. Cloud Music Services
7. Definition of Audio Quality and Happiness
8. Schedule & Sponsors