home  previous   next 
The Twenty-second Annual Interactive Audio Conference
PROJECT BAR-B-Q 2017
BBQ Group Report:
“You and the Uni: Defining Pedagogical Requirements
for Audio Engineering Education”
a.k.a. Discovering What to Learn Them Young Whippersnappers
   
Participants: A.K.A. "Bar-B-Q U"
Brad Lambert, Cirrus Logic Aaron Hipple, Netflix
Miguel Chavez, Analog Devices Peter Eastty, Apple
Kevin Gross, AVA Networks  
   
Facilitator: Linda Law  
  PDF download the PDF
 

Problem Statement

Recent graduates often do not appear to meet our needs as audio engineers. We will define what we think an audio engineering student should know in order to influence their university curriculum choices.

Audio production engineer versus Audio engineer
A quick search for “audio engineering” in The Google yields a long list of for-profit recording/production engineer programs. Although we appreciate and acknowledge the importance of learning how to record and produce music and sound for film, this is not the focus of this report. Rather, the focus of this document is on the engineer designing/building (and testing, supporting, etc.) professional and/or consumer audio electronics.

The larger problem we want to address falls into three broad areas

  1. New hire/college graduates often lack skills we need to design pro/consumer electronics.

    We want to build cool and amazing new audio things - plug-ins, DSPs, augmented-reality and virtual-reality devices, consoles etc. - and we need more talented engineers to help. Recent experience, common to many in the team and corroborated by The Project BBQ participants and alumni in general, is that some new college grads lack the skills we need to build cool and amazing audio products.

    • What skills do we really want?
    • How should prospective students evaluate a school?
    • What interview questions effectively separate the wheat from the chaff?
  1. Existing (non-audio) engineers need a primer/reference to become effective audio engineers

    Audio is a really fun sub-field, so many non-audio engineers get sucked into audio as it affects (afflicts?) the projects they’re working on. A well-intentioned but unskilled engineer can mess up audio and negatively impact customers/users, co-workers, and businesses.

    • What should a non-audio engineer learn in order to make sensible design and implementation decisions?

    In addition to attracting engineers because it is cool, audio benefits from the new ideas others bring to the field. For example, machine learning song classification based on image analysis of spectrograms as presented by Owen Vallis.

    • How do we bring more engineers into audio and The Big Brain?
    • This problem is out of scope for this workgroup.
  1. In addition to technical deficits, more gender and ethnic diversity is needed in audio and engineering in general.

    A perennial problem in STEM and acute in audio in particular: more diversity is wanted.

    • How do we bring under-represented groups into engineering, audio, and The Big Brain?
    • This problem is also out of scope for this workgroup.

Audio things, Really? What products are we building?

  • Hardware and software
  • Consumer electronics
  • Professional audio gear

A brief statement of the group’s solutions to those problems

  1. Outline skills desired for new-hires in audio engineering
    1. Undergraduate Requirements (i.e. entry level) report detailed below
    2. Graduate Requirements (i.e. experienced hire) report detailed below
  2. Primer for experienced non-audio engineers to design audio products
    1. The group did not focus on this aspect of the problem
    2. It is recommended that The Big Brain collate a list of exemplary books, articles, specifications, and courses (the reference corpus) an engineer should refer to for audio education
  3. Increasing diversity in audio and engineering
    1. The group did not focus on this aspect of the problem
    2. Start indoctrination/exposure early targeting underrepresented demographics
    3. Develop audio-themed educational toys and related lesson plans
      1. E.g., lego-like bricks that implement some cool audio thing
 
Bar-B-Q U 

Undergraduate Requirements

These are a set of skills that are fundamental to becoming a professional engineer. If given the choice, we would prefer that someone who is focused on becoming an audio engineer studies these principal topics first. That is, we would prefer the student to do a mechanical/electrical/computer engineering degree containing all the requisite math and physics first. In addition to these fundamental topics, the audio related specialties are to be added. It is much easier for someone with a fundamental understanding of the basics to add the audio-specific details than it is to take someone with an audio focus undergraduate and expect them to quickly assimilate the necessary math and basic sciences. A mastery of engineering fundamentals is crucial for the kinds of independent problem solving a professional audio engineer will face in their practice.


A mastery of fundamentals is the bedrock of a solid understanding

Math

A solid foundation in mathematics is necessary background for a professional aspiring to enter the audio engineering field. A common deficiency seen among candidates is that, during their education, they are given the choice to trade off advanced mathematics curriculum for other, more application-specific studies. Since many audio problems rapidly converge to mathematical problems, a solid foundation is required.

Specifically, an audio engineer must be proficient in algebra, trigonometry, logarithms and exponentials, linear algebra, advanced calculus, differential equations and have experience with time-frequency and other mathematical transformations such as the Laplace, Z and Fourier.

Once advanced math requirements are satisfied, audio-specific topics should be understood including sampling theorem, basic analog and digital signal processing, digital filtering including IIR and FIR techniques, and use of spectrograms for time-frequency analysis.

Lastly it is necessary that the understanding of mathematical topics be followed by computational application using standard tools such as Matlab, Octave, Mathcad, Mathematica and IPython.


Real World/Constrained Problem solving (#Engineering)

One of the fundamental day-to-day activities of an engineer’s work life is problem solving. To that end, we believe it is of the utmost importance that a future engineer engage in engineering labs through their education experience. The earlier they start in the education, and the more they occur the better. The need for proper design hygiene and organization will naturally become apparent throughout these labs. These labs not only ingrain the principals of the scientific method, but they also develop heuristics for efficient troubleshooting and debugging. Furthermore it is more important for future engineers to be able to identify issues within their own work that it is to get the right answer the first time. Labs provide ample experience for making mistakes (and opportunities to fixing them!). As the saying goes, “Measure twice, cut once.”

Additionally lab projects and experiments provide the student with many low- and high-level skills that are necessary in industry.

At the lowest level, labs provide students with the basic constraint of time management in order to meet deadlines. Additionally it gives them access to common industry development and measurement tools. Familiarity with things like creating and reading plots will eventually become invaluable for communicating results. Even the seemingly mundane things like keeping a lab notebook of actions taken during a lab will become critical if a future professional ever has to deal with patent litigation.

At a higher level, labs provide students with systems knowledge that is invaluable for contextualizing results. Only when one understands the workings of the whole system and simultaneously sees how the parts fit together can they really appreciate the interdependencies and know what constraints are material to the problem at hand. Understanding that, in audio, things will eventually “push air” is necessary to know how much noise can be tolerated at the various system stages. Other concerns at higher altitudes of contextualization are the initial exposure to the product design life-cycles. As the product management triangle states, when solving a problem there are three important vectors of Cost, Schedule, and Complexity but one can only choose two.

If the educational program does not provide adequate labs, personal extracurricular projects can provide many of these benefits as long as they are completed to full functionality. Finishing a working design to implementation (even if only a prototype) is considered a fundamental skill.


Interpersonal/Communication Skills

A common complaint heard from hiring managers is candidates often have poor communication skills. While this is by no means a new problem, it is an issue we cannot afford to ignore. Good communication is critical for success of both the individual and the company, individual company projects and career success of individual engineers. Poor communications can, in the extreme, produce dysfunctional teams and flawed designs.

Specific areas of focus identified include basic skills including vocabulary, grammar and verbal communications. Verbal communications in a group context has been identified as a particularly lacking skill that has seen significant degradation in recent years. Interpersonal skills including compassion empathy and cultural sensitivity have been identified as areas requiring additional attention in globalized work environments that feature a decreased opportunity for face-to-face interactions.

Beyond the basics, written communications skills for engineers include the ability to organize technical and design documentation and communicate precisely in these documents, identify and resolve inconsistencies and gaps in specifications. The ability to accurately review and interpret technical specifications and technical standards may be more important for some engineers than the ability to produce such content.

Many engineering projects include a research component often integrated with design and specification development. The ability to do effective research can often identify under-appreciated complications and their associated risks. Effective research in some cases, bring solutions to problems to bear with proportionally little effort.

Improved interviewing skills help the interviewee put their best foot forward and help the interviewer more accurately ascertain the candidate's capabilities. Interviewing skills are also useful in persuasive business and leadership situations beyond the job interview.


Audio specific

Undergraduates contemplating an audio engineering career are expected to have studied core audio engineering topics.

Central to audio engineering is a fundamental understanding of signals and how they operate and are used in engineered audio systems. Important properties of audio signals are amplitude measured in various ways including peak amplitude, root-mean squared and delivered power.

A complete understanding of signal and system performance measurement techniques and metrics is essential in the characterization of signals and systems that use them. A familiarity with logarithmic scales for frequency (i.e. octaves, decades) and amplitude (i.e. decibels) are a requirement in aspiring candidates.

Signals in an audio system can be characterized with mathematics and measurements so a comprehensive competence with these representations is required. This requirement includes the ability to understand and interpret common audio test plots and specifications.

It is important to recognize that all audio systems are eventually associated with the interactions with air via a transducer and therefore listening is fundamental to audio engineering. Students of audio engineering are expected to develop some measure of listening skills including familiarity with a reference system and how listener training works. A basic understanding of music, sound reinforcement and recording and production processes is helpful in many audio engineering roles.

In many applications, audio’s interaction with the world is an important part of the engineering problem. As such, a good understanding of physical acoustics principles is fundamental.


Computer science

Gone are the days of pure analog synthesis and tight digital design. Today’s audio widgets demand an understanding - no, a command - of computer science to be effective, compelling, and scalable. The computer science background required for audio is in some ways one of our most demanding requests, because the need is both broad and deep. Key topics include:

  • User experience (UX) and user interface (UI) design
  • Object oriented programming
  • Algorithm development
  • Common design patterns
  • Process threading
  • Hardware description language (HDL)
  • Software architecture and abstraction layers
  • Solution scalability
  • Optimization for design targets

UX and UI design are more than buzzwords; good design makes products easier to use and more engaging. Good design starts with the understanding that rigorous research has already identified effective UI design (see “Interaction Design - 4th ed.” by Preece, Rogers, Sharp; “The UX book” by Harrison & Pyla; and “Sketching User Experiences” by Buxton). Applicants are strongly encouraged to develop a familiarity with common UI elements of digital audio workstations (DAWS) or audio editing software. Applicants are additionally reminded that just because they can make a user interface in Labview, Labview is not paragon of UI design.

Object-oriented programming is an expected skill for any applicant; however, mastery of a specific language is not required (although a specific language may be required at particular companies). Applicants are expected to be familiar with required, institution-specific coding style/standards (i.e., accept that work products must adhere to company standards) and development for maintainability.

Algorithms are fundamental to a computer science education. Mastery of bread-and-butter algorithms such as data structures, streaming and memory management give the candidate a leg up in special adaptations of these algorithms to audio applications.

We want to solve interesting problems in audio. Many interesting problems already have significantly developed solutions. The effective engineer familiarizes themselves with previous work in the problem area including any applicable design patterns. By leveraging previous work and common approaches to similar problems, we are able to concentrate of the truly interesting parts of a problem and more quickly deliver robust and mature solutions.

Bare metal and assembly programming are no longer a best-practice approach for general programming. However, familiarity with underlying hardware architectures and targeted optimization of code segments (potentially in assembly) is a desired skill. The applicant should note that optimization may take many forms; it may mean operating at a lower power consumption or in a shorter period of time. Optimization is fitting the work product into a smaller operational envelope. On the other hand, solutions must also scale to larger deployments. Although we do not expect undergraduates to have completed examples of large, distributed computing applications, they should be aware of how cloud computing works and understand best practices for scalability.

Although they may not have completed a large scale multi-threaded application, the applicant should be familiar with process threading. The applicant is recommended to understand mutexs, process preemption, and common scheduling schemes. Undergraduates are not expected to have developed examples of these mechanisms from first principles; however, they should know how to use them.

Modern digital circuits are designed using hardware description languages (HDL), and an applicant heading into the integrated circuit industry should have a basic understanding of HDL. Like software programming, an undergraduate applicant should be aware of institution-specific style requirements and design for synthesis guidelines.

We have described requirements ranging from HDL synthesis to maintainable object-oriented programming; this is a broad scope. On top of this, a savvy applicant should understand the software architecture and abstraction layers that build complicated (read as interesting) software systems. For example, an applicant would be well served by being able to trace the path of audio through a modern OS from application to analog.


Graduate Requirements

These are skills we consider high level study requirements. They need not be obtained in a graduate program, but they are typically expected of someone filling a more advanced role.


The wisest wizards live in the tallest towers

Math for Signal Processing

Mathematics are fundamental to engineering. By the time an applicant reaches the graduate level all the basics identified for undergraduates should be second nature. Mathematics for signal processing involves tailoring mathematical knowledge to computational possibilities. It is not sufficient to be able to solve a problem in MatLab, the applicant needs to be able to take that solution and implement it as a real-time solution on a real processor. Implementation involves dealing with the practicalities of what can be computed within allowed engineering constraints and real-world considerations.

In the adaption of algorithms to an implementation, issues such as floating vs. fixed point computation and appropriate bit depth for specific computations must be considered. The audio engineer must know techniques that can be used to resolve these questions at a practical level.

Not all engineering problems can be understood as textbook linear and time invariant systems. In particular, adaptive processing (e.g. echo cancellation) requires that multi-variable optimizations are performed in real time under varying input stimuli. The graduate level applicant who has not met adaptive processing cannot appreciate many facets of modern consumer audio electronics.

Modern audio products are systems comprising multiple interconnected processing units. These systems often include sample-rate conversion. A capable audio engineer will have a good understanding of both the theoretical underpinning and practical considerations of sample rate conversion.

Delta-sigma modulation is a process frequently used in analog-to-digital and digital-to-analog conversion and is also sometimes used inside digital-only systems. The mathematics of what are essentially stochastic processes have not been fully developed, but a well-trained audio engineer should be familiar with what is known.


Computer Science

The missing elements in a computer science education when applied to advanced audio engineering relate chiefly to time, involving synchronization and real-time processing constraints. Audio processing by computers and signal processors involves timing constraints measured in tens of microseconds. The tens-of-milliseconds timescales of video and graphics processing and the hundreds of milliseconds in internet processing do not prepare young engineers for the timescale of audio processes. Delivering a sample on time is just as important as delivering the right data.


Interpersonal/Communication Skills

Interpersonal courses like those taught in MBA and/or executive courses are highly beneficial to young professionals. Exercises that demand that the student is placed in other people’s shoes or, a different position within an organization are useful. These enhance team building, communication and interpersonal skills and provide significant benefits. Other types of exercises required by young professionals are needed to understand and ingrain the concept of empathy both within individuals, in an organization, and with end customers. Empathic individuals clearly understand that they are not the user base, and are always open to receive feedback. These skills ultimately result in better, more successful products.

Another identified problem with engineers in the ever-so-complex and evolving engineering in today’s world is to be able to assess when to continue existing work and when to re-engineer a solution. Engineers must be comfortable both reinventing and taking over existing projects, and that this decision must be carefully considered.


Audio specific

And finally, we come to the audio-specific knowledge that is required for an advanced applicant. This is generally considered “the fun stuff” and a good deal of it may be learned from books or online resources.  At a basic level, the advanced engineer should have a very good knowledge of audio systems.  For example, they ought to be able to layout the block diagram of a portable music playback device including the appropriate blocks (i.e. storage, CPU, decoder, DAC etc).  Necessarily this level of understanding includes knowledge of the properties and use of common audio transducers like microphones and speakers.  For certain mechanical engineering jobs, this expertise would also include advanced physical acoustics knowledge including how to calculate lump sum parameters, finite element analysis, or speaker design. In electrical engineering specialties, it may require knowledge about proper grounding techniques and how to maintain analog signal integrity.

In addition to the “hard” science audio topics mentioned above, the related and very important “soft” science topic of human psychology of hearing is critical. In perceptual audio coding, temporal and frequency masking are obviously fundamental tools used to achieve data compression. Many audio engineers must explain the delicate relationship of amplitude to the human perception of loudness to non-experts, so precise knowledge and language are required.  Understanding of topics like the equal loudness contours, bark bands, missing fundamental, and basic working of the ear are often useful. In certain multimedia fields it is beneficial to know esoteric things about the interplay between video and audio (e.g. McGurk effect) but always to keep in mind the importance of latency including lip (or audio/video) sync.  Knowledge of psychoacoustics is necessary to make the best trade-offs in audio system design.  For example, based on a system’s final sonic output level and expected masking, design trade offs in the selection of audio bus bit depth can be made. That is, you must appropriately consider the entire system specifications relative to both the objective engineering (like signal to noise ratio) and subjective human vectors (like signal to mask ratio).


Real World/Constrained Problem solving (#Engineering)

While we recognize that labs are expensive, they’re also highly valuable and must be a central part of both undergraduate and advanced engineering curricula. An engineer without practical skills is severely handicapped in the work they can be assigned. Skills associated with lab experience were one of the most frequently-identified deficiencies in applicants.

Proficient candidates are able to develop from first principles (e.g. write an FFT in C or similar) not just invoke a library function. They are able to understand basic audio protocols like SPDIF, timecode and wireless and wired networking basics. Audio engineers need to be able to write high-performance code at the lowest levels of a computing platform using optimization and taking into account memory constraints, cache performance, MIPS budgets, and latency constraints.

There’s a strong call among hiring managers for interdisciplinary knowledge; software people understanding how to control hardware, hardware people understanding what makes the software engineer’s job easier and understanding both analogue as well as digital hardware. Most importantly, all developers need to have a good understanding of the end application or product to achieve natural alignment.

Designs needs to accommodate real-world constraints. Many audio-enabled devices run on batteries. Low-power design is in every electronic engineer’s future. Engineers also need to design with an eye on cost; engineering was once defined as “doing for 10¢ what any fool can do for a dollar” -Duke of Wellington Arthur Wellesley.

Intellectual property law is much more invasive part of engineering than it was decades ago. A post-graduate engineer should know how to behave with regards to IP law without landing the company or themselves in court.

Internships are amazingly useful for all parties. The student gets real world experience and, in many cases, a permanent job. The employer gets young ideas and energy. The employer’s association with the university can improve its attractiveness to prospective students and may produce research funding for the university from the employer.


section 4


next section

select a section:
1. Introduction
2. Workgroup Reports Overview
3. Alexa, Siri, Cortana or: How I Learned to Stop Worrying and Love the Cloud
4. “You and the Uni: Defining Pedagogical Requirements for Audio Engineering Education” a.k.a. Discovering What to Learn Them Young Whippersnappers
5. A spatial audio format with 6 Degrees of Freedom
6. CAAML: Creative Audio Applications of Machine Learning
7. Mode and Nodes Enabling Consumer Use of Heterogeneous Wireless Speaker Devices
8. Abusing Technology for Creative Purposes
9. Schedule & Sponsors