Spatialized Audio in a Virtual World

This is a classic audio clip that we are all big fans of here… the famed Virtual Barber Shop.

As we started to think about designing the audio for our new world, we listened to that clip and asked ourselves why it shouldn’t be possible to create effects that lifelike, or even better, in a virtual environment? The combination of network bandwidth and device processing power makes it feasible to have a virtual world where one or more audio ‘servers’ can do the work of computing a stereo signal to be fed to each ear of the listener. If we are also getting low-latency data about the exact rotation of the listener’s head, we can do a lot of amazing things, and can ultimately reproduce all of these effects.

For example, sound in the real world arrives at a slightly different time at each ear, depending on where the sound was coming from (and this effect is also different depending on the frequency of the sound). This difference of a bit less than one millisecond corresponds to 30 or so samples in a 44Khz audio playback system, meaning that the effect can be easily be created by shifting the left and right streams by a few samples in each direction. And this effect works quite nicely, as the clip above illustrates. What the clip cannot demonstrate, though, is how compelling the effect is when it is produced by turning one’s own head (whose rotation we capture in our lab with gyros attached to the frame of a pair of glasses).

We’re working on combining a number of effects like these, as well as creating support for a very large number of simultaneous sources of audio, to hopefully create an audio ‘environment’ that will be an interesting to explore as the real world.