home  previous   next 
The Fourteenth Annual Interactive Audio Conference
PROJECT BAR-B-Q 2009
brainstorming graphic

Group Report: Re-imagining Operating System/Hardware Services for Applications

   
Participants: A.K.A. "Control Hierarchy for Understanding Devices"

Dan Bogard, IDT

Niel Warren, Apple

Jim Rippie, Invisible Industries

 
 

Brief statement of the problem(s) on which the group worked:

Nobody’s happy with the current audio recording and playback attributes advertised by the platform; ideally, all the “building blocks” of the overall solutions (for listening to music, playing games, messing with audio, doing music for film, etc.) should know about each others’ capabilities in order to provide users experiences that meet expectations.

The platforms have to guess at the applications’ needs, usually grasping in the dark and estimating. The OS lies to the Apps and the Apps lie to the OS regularly; the same is true of audio subsystems and end points.

Limitations in operating system and standardized audio hardware and external audio interconnect standards should be improved, or replaced.


A brief statement of the group’s solutions to those problems

To address this problem, we established some selection criteria for checking the the success of an audio subsystem on next generation products:

  • Capability discovery
    • Static
      • number of channels
    • Current
      • My latency right now is X
  • Configuration (How do apps configure capabilities?)
    • i.e., how does a VOIP app configure noise management, echo cancellation, beam steering, etc.? microphone configuration (cardioid, omni, etc.)? Even VOLUME and MUTE?
  • Subsystem virtualization

We also discussed the benefit of introducing  “Device Classes” (common in some interface standards) so that audio endpoints, audio subsystems, OS services, and applications can self-describe a rich set of attributes and provide mechanisms to configure and control behaviors. We’ll pursue and flesh out this idea, and propose a baseline set of attributes when finalizing the report in the final weeks.

The action item list

 

Who’s Responsible

Due Date

Description

1

Jim

11/25/2009

Complete report for publication

2

Dan

11/9/2009

Provide a list of common attribute for several audio endpoint (microphone arrays, DACs)

3

Niel

11/9/2009

Provide a set of common attributes for: speakers; clocking; Apple’s Audio Units); and simple audio end user application


Expanded problem statement:

The HD Audio subsystem standard, benefits and limitations:

HD audio did solve major problems:

  • Multiplexes control and data on the same link, big advantage over earlier technologies
  • Allowed sample rates beyond 96k and channel counts >6
  • Lots of other advantages over AC97 and other contemporary/earlier
  • Architecturally simple from hardware system perspective; simple = elegant

Unfortunately, during HD Audio’s definition and implementation phases, developers either deliberately postponed addressing some problems, or created some new issues:

  • What problems (or opportunities) were intentionally set aside or ignored?
    • All integration issues except how to integrate a codec
      • Not how to create an audio system for an integrated solution (including configurability of parameters related to acoustics)
    • Peer to peer communication, including peer-to-peer clock sync
    • Does not support bridging (intentional)
      • Among the issues, Does not support audio routing
  • New problems HD introduced
    • Cannot be synchronized to an outside source, actually cannot be synchronized with itself
      • Doesn’t acknowledge the existence of other audio transports
    • Power Management
      • The bus isn’t manageable (can’t be tuned up or down)
        • E.g., prevents it from being applicable to UltraMobile, would require adjustable clock rate
    • Problematic Extensibility
      • The spec’s extensibility contracts the original goal of the standard: reduce pin count
      • For all existing implementations (due to limitations in the controllers), multiples streams across multi-channel audio endpoints doesn’t exist (even now, nobody makes the controllers)
      • it’s horribly inefficient to extend (nobody does striping, not even vendors of HDMI controllers that spoof the host into believing that it exists)
      • Extending bandwidth by adding more lines actually perpetuates the problem HD was trying to solve initially
        • E.g., moving from stereo to 3 or 4 channel on I2S means adding lines
    • Definition of the data structure
      • Static structures trying in vain to define dynamic (does not hit high channel counts for pro applications)
    • Doesn’t support multiple masters
    • Does not support phase-accurate aggregation of endpoints (e.g., two stereo codecs looking like one four-channel  input/output)
      • Perhaps due to assumption that a higher layer would manage the phase relationships, documentation in this area is too ambiguous
      • The “delay” (processing delay) value is a Parameter, set at chip design time; its resolution is too low, it’s often inaccurate, and it can’t be changed
      • E.g., Phase alignment for multiple speakers
    • Ignores Calibration and Configuration info
      • Where is it stored? It’s not available via HD

Also, big problems still exist, since nobody foresaw them when they defined HD Audio:

  • Figuring out a sensible way to control the routing and logical configuration
  • A closed ecosystem means integrators have to fight it
    • Because the parts aren’t building blocks, integrating them requires access to documentation that usually doesn’t exist (or that key parties withhold)


Expanded solution description:

Any new industry solution that solves these problems:

  • Should address highest priority
    • Problems HD audio chose to ignore
    • Problems HD audio didn’t think about
    • Problems HD audio created
    • Next-generation ass covering features
  • Should “draw a big fat, grey, blurry line” between the bottom of the application and the top of the Operating System and let each layer of the solution describe key attributes (and where appropriate configure them across layers)


Items from the brainstorming lists that the group thought were worth reporting:

  • Applications want to know
    • Timing/Clocking
      • Roster of available clocks and clock domains
      • Clocking services including
      • clock routing
        • i.e., domain management
    • Throttling status (e.g., temp sensor warning causes clock scaling)
    • Number of processors
    • Count, Locations and types of transducers
  • Operating Systems want to know
    • thread priority
      • including latency requirements
    • ensitivity to page replacement algorithm
    • some notion of “Application class” (c.f. USB device classes)
      • are you a chat app (very latency tolerant), or a telephony app (very latency neurotic)

section 6


next section

select a section:
1. Introduction
2. Executive Summary
3. Hear, There, and Everywhere
4. Dick and Jane Tracy plug into the Matrix
5. Mobile Infra-Structure
6. Re-imagining Operating System/Hardware Services for Applications
7. Schedule & Sponsors