Although we won’t be exhibiting at IBC 2017, 3D Sound Labs technology will be showcased on some partners’ booths
Paris – September 14, 2017 – Dimitri SINGER – Co-founder & CEO – 3D Sound Labs
Later this week, and like every year, all the broadcast industry will gather in Amsterdam for the IBC, this year the IBC 2017, the professional trade show that calls itself the “world’s leading media, entertainment & technology show”. Indeed, IBC is arguably the most influential annual event for professionals engaged in the creation, management and delivery of electronic media, worldwide.
The combination of 3D Sound Labs’ head tracking technology and Microsoft’s 3D Audio and integrated system design expertise makes it possible to create more powerful and immersive user experiences.
Over the years, the focus has shifted, or rather expanded beyond TV and cinema, to include internet-based media and interactive content such as 360° video and virtual reality.
With those new media come new challenges for the industry, some directly related to audio and some that are more general but that create strong constraints on the way audio needs to be processed :
- 3D sound recording and mixing requires special equipment and tools: ambisonics microphones, that record a 3D sound field rather than a single sound source and specific software for the sound designer to be able to place and move sound sources in 3D space and listen to the result in real-time. Until recently, this has been lacking.
- Delivering complex spatial audio over “standard” file or stream formats can be tricky: an object-based description, where each sound-emitting-object is described separately could require a tremendous number of audio channels for complex sound scenes. Using an ambisonics “sound field” description lowers the number of channels required but when higher order ambisonics is used to increase the sound’s spatial resolution, the limitations of industry’s current audio encoders are reached and new codecs are needed
- The interactive nature of VR and 360° media, where the user can decide where to look, adds strong requirements for the streaming of the video and the processing of the audio rendering: streaming high-quality video at 360° when a person’s field of view is limited is a waste of bandwidth. To lower the bandwidth requirements for high quality content, a special pre-processing called tiling has been introduced, where only the portion of video corresponding to the field of view is streamed. It requires a complex low latency feedback mechanism between the player and the server. User’s interaction with the content also requires low-latency real-time audio scene rotation, controlled with the head tracking. As standard multimedia players usually buffer up to 20 seconds of audio to offer a good QoS while coping with streaming glitches, this means that a new software architecture needs to be implemented at the player side.
3D Sound labs is addressing those industry challenges and helps accelerate the deployment of solutions for the VR and 360° video markets:
- Since 2016, 3D Sound Labs is part of the European BINCI consortium, encompassing all the different expertise areas involved in the 3D Audio creative workflow: recording production, post-production, exhibition and delivery. The BINCI project (which stands for binaural tools for the creative industries) received funding from the European Union’s Horizon 2020 research and innovation programme. The goal is to deliver market ready solutions proven in real production environments by early 2018.
- We believe that ambisonics is the description of choice for spatial audio scenes. Ambisonics is the obvious native format of 360° recorded content. It also brings scalability, to adapt the content to the available bandwidth or CPU constraints, especially on mobile, by considering only data up to a given ambisonics order. Encoding first order ambisonics (the so-called FOA or B-Format) is possible with current AAC codecs. To deliver higher order ambisonics content, 3D Sound Labs is pushing for the adoption of new audio codecs such as Opus. Opus is an open, royalty-free audio codec compatible with ambisonics since recent extensions to the Opus format allow for the transmission of ambisonics audio with various parameters.
- 3D Sound Labs audio rendering technology is available for licensing through VR Audio Kit, the company’s SDK. The VR Audio Kit binaural engine works with any sound scene: object-based, multichannel beds and ambisonics. It renders audio over any pair of standard headphones or earbuds, with an extremely low-latency sound scene rotation for headtracking. Its internal processing, based on a high-order-ambisonics representation, is optimized for sound quality and CPU consumption and delivers industry’s leading performances on multiple computing, mobile and embedded platforms. A high-efficiency, proprietary set of HRTF (or Head-Related Transfer Function, the morphologically dependant audio filters that help the brain locate sound in 3D space) is provided with VR Audio Kit, and soon the user will be able to use its own, personalized HRTF thanks to our upcoming HRTF service which is the outcome of several years of internal research in this field.
- 3D Sound Labs has also developed an audio plugin for Unity. The tool lets content creators render any object-based (and soon ambisonics) audio scene. With Unity, they can develop desktop and mobile applications and unleash the potential of 3D Audio for games and virtual reality apps.
Although 3D Sound Labs will not be exhibiting at IBC 2017 this year, we will be there (and willing to meet if you’re interested, please get in touch) and our technology will be showcased at some of our partner’s booths :
- On Harmonic booth (1.B20), get a taste of Blue Man Group live at the Luxor Hotel and Casino in Las Vegas in a unique VR/360 experience. Produced in collaboration with Digital Immersion and powered by Harmonic, this live performance brings you up close and personal with the Blue Men in stunning 8K. This demo of an end-to-end solution is powered by Tiledmedia and Viaccess-Orca mobile client that embeds 3D audio processing by 3D Sound Labs.
- NexPlayer (booth 14.D24), a market leader Player SDK for Android, iOS, Tizen and HTML5 apps will showcase a 360 video demo with NexPlayer and 3D Sound Labs. The mobile video player powered by NexPlayer260 SDK and VR Audio Kit is compatible with ambisonics audio content as well as higher-order ambisonics for better audio spatialization. The NexPlayer360 SDK is available for evaluation on their website.
At 3D Sound Labs, we are convinced that 360°/VR/AR/MR (in whatever form or name you prefer) will be the next wave of digital media and entertainment because it brings an extra level of immersion, and freedom to explore the content. Our goal, through our research and developments, is to provide the most advanced 3D audio technologies to increase realism and provide customers with the best possible experience.