3 Surprising Use Cases for 3D Audio

3D audio has become one of the loudest trends this year — and its impact and reach has been vast.

Of course, this type of audio is not new: 3D sound has been used to create immersive experiences in film, gaming, and virtual reality for years.

“By inserting 3D audio, new spatial information is introduced to the virtual experience, enabling audiences to sense things happening behind them, or elsewhere in their virtual environment, completely independent of their eyes. Just imagine yourself at a museum, standing in front of a massive painting that – despite being a two dimensional work of art – has amazing depth and allure. Visually, it draws you in and creates a sense of rapture. Now fold 3D audio into the experience with carefully crafted sound that wraps itself around you and pulls you closer.”

Here’s another example: “If you’ve played Resident Evil 2 using just a pair of regular stereo headphones, you’ll know that if Mr. X is stomping around in his massive boots on the floor above, you can tell he’s one storey higher thanks to the way the noise sounds. And that’s not because of the thuds being muffled or disguised to simulate distance; the noise quite literally ‘feels’ like it is coming from above. That’s 3D audio in action.” 

True — the boom in next-generation consoles with 3D audio has changed the gaming experience. But as we mentioned, the trend of implementing 3D audio has been extensive, and this type of sound is now being integrated into many more experiences than you might have even realized.

For example, have you used Clubhouse, the social audio mobile app, recently? A few months ago, they integrated spatial audio to make users feel like they were actually in the room together. (More on this below…)

Although you likely immediately see why having immersive audio in popular types of 3D virtual events (such as concerts and conferences) makes sense, let’s talk today about use cases that might not be as obvious: Ones that arise from 2D platforms.

3 Surprising Ways To Leverage 3D Audio

Among surprising ways to leverage 3D audio, 2D platforms may not first come to mind due to being… well, flat. Of course 3D audio makes sense in 3D online environments, but 2D?

But let’s consider the sort of virtual experience that various 2D platforms are creating between humans: Ones that are driven largely by talking to one another for connection. Do you know that audio-only interactions are actually easier for humans to comprehend emotionally than using video?

By the end of this post, you’ll see why 3D audio is just as important for 2D platforms to leverage, too.

1. Proximity Chat Platforms

Early in 2020, an article was published on WIRED Called A Mission to Make Virtual Parties Actually Fun.

Gretchen McCulloch writes, “We’ve been in this pandemic for a while now, and we've sort of gotten used to a primarily virtual social life. Meetings and classes on Zoom are adequate, the catch-up call while you tidy your kitchen works quite well, and the coworking video with the sound off is surprisingly effective. I've actually enjoyed being able to attend panels with people who otherwise wouldn't be able to be in the same location. But there’s one thing that still glaringly doesn't work—the virtual party.”

She goes on to describe her mission of finding something that mimics a party in a more real way, and finding proximity chat platforms. “They’re an emerging genre of communication platform that's halfway between Zoom breakout rooms and chatting during video games. The basic idea is that you have an avatar or icon that can walk around virtual spaces or jump between virtual ‘tables,’ and, much like in real life, you can see and hear only the people who are nearby. This allows people to move themselves fluidly between conversational groups while still having a shared sense of the whole room.”

And the above is precisely why 3D audio is so important for these 2D platforms — so participants are immersed in the environment together, and the “cocktail party effect” is in full effect.

For example, check out Skittish, a virtual browser-based event platform. “A playful cross between a social audio chat app like Discord or Clubhouse and a cute video game.” They integrated spatial audio earlier this year from High Fidelity.

Curious to learn more about proximity chat platforms? Read more here.

2. Social Audio Apps

We mentioned Clubhouse briefly above, and will go into more detail about the social audio app here. However, it should be noted that social audio as a trend is growing hugely, and there are many more platforms popping up (including every major social network integrating social audio features).

When Clubhouse integrated spatial audio, Taylor Hatmaker interviewed their Head of Streaming, Justin Uberti, about the details. “While Clubhouse and other voice chat apps bring people together in virtual social settings, the audio generally sounds relatively flat, like it’s emanating from a single central location. But at the in-person gatherings Clubhouse is meant to simulate, you’ll be hearing audio from all around the room, from the left and right of a stage to the various locations in the audience where speakers might ask their questions.”

Hatmaker continues: “To pull off the new audio tricks, Clubhouse is integrating licensed code from Second Life creator Philip Rosedale’s spatial audio company High Fidelity and blending it with the company’s own custom audio processing, tuned for the chat app. High Fidelity’s HRTF technology, which stands for ‘Head Related Transfer Function,’ maps speech to different virtual locations by subtly adding a time delay between stereo channels and replicating the way that high and low frequencies would sound entering the ear depending on a sound’s origin.”

How does immersive audio help the 2D app? “There are a variety of specific scenarios spatial audio can enhance. Experiences like comedy, music and even ASMR have great subcommunities on Clubhouse that can benefit. ‘On Clubhouse, when you feel the laughter come from all around you, it feels a lot like a comedy club experience,’ explains Uberti. Someone telling a joke won’t feel flat.”

Indeed — Clubhouse has now enabled millions of people to participate in various rooms that create the experience of being in the room with others (enjoying a live audio musical like the Lion King, or an educational talk, for example) even while using a 2D platform.

3. Videoconferencing Software

What if videoconferencing calls weren’t so… exhausting? Zoom fatigue has been well researched at this point.

Stanford University’s Jeremy Bailenson explains: "Part of the reason we're fatigued by all these Zoom meetings isn't just the presence of video, it is also the audio quality and how voices are delivered to our ears. Think about it: in the physical world, all the sounds we hear and process in our brain come from different locations in a 3D environment. When you're at a cocktail party, you can have multiple people talking at the same time and still understand what everyone is saying.” There’s the importance of the cocktail party effect again. Read more about the future of videoconferencing here for more details.

3D audio helps these 2D platforms by decreasing cognitive load as well as improving speech intelligibility. Check out more of this research here.

Each of these 2D apps is massively improved by integrating 3D audio. If you’re working on a native app (like Clubhouse, for example), you can implement spatial audio easily now with the Local Spatializer. And if you have a web app, check out the API.