Why are music visualizers so artless? Most look like hardware spectrum analyzers or jiggling oscilloscopes. Perhaps it’s because the input data is limited to snapshots of bass and treble levels.
Let’s explore ideas for extracting emotionally meaningful information from audio to produce next-gen visual experiences. Imagine if a screen, garment, or virtual-reality experience could react to chord quality (e.g., major/minor/augmented), subtle tempo changes, solos, or even lyrics. A combination of deeper audio analysis and metadata could create experiences as exciting as movies and live dancers.
Untold millions of Internet-connected devices are slacking off when they could be working together to solve Big Audio Problems. The Search for Extraterrestrial Intelligence (SETI) program harnessed unused cycles on thousands of computers to hunt for aliens. What kinds of audio problems could we solve by networking millions of underused devices? What new sonic experiences could we create?
Now imagine extending the processing pool to mobile devices with microphones and location sensors. What are the opportunities for Massively Multiplayer Music? What incentives would inspire people to share access to their processors and I/O?
Transmission systems: What are the file/transport format requirements?
Composing and synthesizing systems
I’ve listed these in reverse to imagine the payoff so we can identify ways to get there.
As a background question, what are some Joe-friendly ways to record and distribute immersive audio today? I’ve tried software to make “AC3” DVDs and DTS CDs, the Zoom H2N 5-mic recorder’s stereo mixdown, in-ear mics, and spatializer plugins, but none has really curled my mustache.
YouTube is now the largest music-streaming service. Facebook lets you post videos, but not audio. Many musicians get around this by posting non-moving movies consisting of a stereo music track and a picture of an album cover. But what if music could become the foundation for dynamic, compelling video? That could make the whole audio chain more popular.
Imagine a visualizer driven by a combination of audio metadata, DSP, and artificial intelligence…perhaps even influenced by other sensor inputs. Instead of wiggling wireframes, this system could approach cinematic storytelling. And not just in video, but AR and VR as well.
What hooks could we add to audio files to generate more immersive visuals? What are the opportunities in production and delivery? And why are people who imagine the future called visionaries?
Footnote: Creative Labs did some groundbreaking work on music visualization back in 1999 with Lava/Oozic. The system used a proprietary file format and web player, and it died around the dot-com crash, but there were some ambitious ideas in there.
In the article, Peter, who’s also a two-time BBQ speaker, shares his insights on adding warmth and personality to devices through evocative sound. In the emerging Internet of Things, imagine how much further that could go if devices not only sung beautifully, but also harmonized with each other and the environment.
What’s the hottest new consumer technology? Helicopter drone video. What’s the worst thing about helicopter drone video? That droning sound. (Or no sound at all.) Imagine…
A drone-mounted mic that cancels the propeller sound, producing pristine soundtracks
A ring of drone-mounted speakers that follow you around, for mobile surround sound (“wingtones”)
An app that synthesizes music from silent drone video
More practically, future noise cancellation algorithms will offer numerous opportunities for adding sound and music to previously hostile environments. What are some scenarios that would encourage that development?
The New York Timesreported that Google Fiber “is so fast, it’s hard to know what to do with it.” After downloading 612 kitten photos in one second, researchers wondered what to do next with the gigabit connection.
Could the killer app involve audio? As gigabit speeds roll out across America, what are the audio opportunities?
It saddens me to see two people sharing a single set of earbuds, one lonely channel per listener. What if there were an app that let one person broadcast music to the mobile devices of nearby friends? Add voice and text chat, plus some ways to monetize the service through DSP plug-ins, hardware add-ons, and referral fees for promoting music.
What are the requirements, and what types of businesses could result?
Failing that, how about some really immersive headphones?