While GPUs have been ubiquitous for a long time in industries ranging from artificial intelligence to gaming, the concept of utilizing graphics cards for audio processing is still relatively new. There were a number of fundamental reasons related to computer science, mathematics, and sequential DSP algorithm design.

To meet a cross-industry need that arose given the advances in accelerated computing, spatial audio processing, game audio, machine learning, and AI — to name a few — GPU Audio set out to develop a viable industry standard for real-time professional GPU audio processing. GPU Audio has become the first company to develop the world’s first full-stack solution for GPU-based technology for audio processing, solving fundamental problems by embedding the scheduler to building own ways to operate at low latencies, re-inventing the math to describe the traditional algorithms and be able to migrate them for the GPU architecture, and more.

As Andy Cook, Senior Manager at NVIDIA’s Deep Learning Institute (DLI), explains, much of his career was based in audio. Coming out of college, Cook started with Digidesign, developing Pro Tools, which was later acquired by Avid. “Seeing this transition from analog workflows to digital [ones] and the impact, new techniques, and opportunities that came out of that transformation was really exciting,” he says. “Along the way, you also recognize that things like the Convolution Reverb are so demanding — the DSP horsepower required for really putting that into practice was very limited.”

Moving to NVIDIA, Andy noticed that on one side, there was a lot of demand for audio processing power, and incredible processing power on the other — so, he was waiting for the bridge that could connect the two. “When GPU Audio came along, it was a hallelujah moment because now you see this combination of groundbreaking innovation opportunities that are now connecting. I think it’s really exciting for the future,” Andy concludes.

A big part of the startup’s vision was creating a software development kit, introducing new opportunities for the developer community. Having worked on it for months, GPU Audio will be presenting the SDK for the first time in a DLI Training Lab at the NVIDIA GTC 2022 Conference. Through the course, developers have the chance to learn about the foundational issues solved by the new GPU Audio standard, dive deeper into its core technology, and learn how to incorporate real-time/low-latency DSP algorithms into their projects.

“Rolling out the SDK is indeed a big milestone for us, as this will allow us to share our technology with the other players in the audio industry, giving them the possibility to enhance their products with our GPU-powered engine,” Andrés Ezequiel Viso, the GPU Development Product Manager at GPU Audio, begins, saying that it’s a big step forward towards the company’s aim of making GPU Audio a new standard for professional audio processing. Alexander Prokopchuk, GPU Audio’s Chief Technical Officer, continues by saying that the SDK ties all of GPU Audio’s work being done over the last five years and materializes it into the reusable platform that can be utilized to create GPU-powered pro audio applications.

“I’m very happy that our vision and our work led us to this moment and we’re ready to release a preview of the SDK,” Prokopchuk says.

The software development kit was designed for audio effect developers. As Andrés says, GPU Audio aims to not only collaborate with the big companies in the field but also invite audio enthusiasts to develop their own GPU-powered effects. “In that sense, we wanted the SDK to adapt to so many different possible scenarios while still being something accessible for developers that might not necessarily come from the GPU background,” he shares. Given that, one of the challenges was keeping it simple and accessible while not compromising its potential.

Rolling out the content kit involved a myriad of challenges coming from different fronts. Developing the GPU Audio solution was a monumental task that required reimagining the whole workflow of audio processing — everything from DAW communication all the way to low-level GPU optimizations. Once the GPU Audio team came up with the very first working prototype, the challenge was to strip it down into different self-contained components that made sense on their own. This process guided the design not only of the SDK but of the overall solution. “In a way, the first users of the SDK were ourselves when dividing responsibilities inside the company itself,” Andrés recounts. “We arrived at this point with something that was not only conceived as an SDK from its design stages but also iterated a bunch of times as part of the development process of our own effects, like the EAP or modulation bundle we are releasing soon.”

Sasha adds that the main challenge was taking those different parts and assembling them into one entity. “One of our goals was to not let the internal complexity of the GPU for audio processing problem affect the complexity of SDK APIs,” he says, explaining that the company wants to provide an easy-to-use kit, and for that, the team had to thoroughly think through all APIs, scenarios, documentation, examples, and more.

The workshop with NVIDIA’s Deep Learning Institute — which will be held at the 2022 GTC Conference — will be the very first time GPU Audio shares its SDK with developers outside of the company. “The idea of the workshop is to serve as a developer’s introduction to GPU Audio, sharing the main concepts behind it, explaining the workflow behind the SDK, and giving an overview of the possibilities behind the technology,” Andrés conveys. Ideally, anyone attending the workshop will leave it with their own simple working effect and a clear understanding of how the different parts of the solution interact and what can be accomplished.

For Andy Cook, this workshop is a great way to provide a hands-on experience, where they would actually roll up their sleeves and get into the nitty-gritty of it. “A lot of people recognize the enormous potential for GPUs in audio but may have shied away because of the challenges associated with harnessing that power,” he explains. “This training lab is a great opportunity for developers to see new possibilities in this area.”

On the other hand, it’s also important to get user feedback about the kit and how easy or hard it is for new developers to understand its interfaces and structures, the workflow, and more. From there, this potential feedback will be taken into account to improve the SDK for future versions.

As Andrés explains, the software development kit will be released in several layers. “We are first rolling out the NVIDIA version of what we call the ‘processor_api’, with this component being designed to give the plugin developers the means to communicate with our core engine component,” he elucidates. At the same time, although not covered during the coming DLI Training Lab, GPU Audio is releasing the ‘dsp_api’, which handles the other end of the communication between the engine and the DAW.

Another layer of the SDK that will be released afterward is the GPU Audio custom DSP library, which will comprise a set of data structures and algorithms for signal processing specifically designed for graphics cards. The idea behind it is to provide the user with some commonly used building blocks and accomplish more complex and interesting effects without having to worry about redesigning them to fit the GPU architecture.

The DLI Training Lab will be taking place on September 21, 7–9am PDT. Looking forward to seeing you there: https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus&search=DLIT41283#/session/1657231314808001VoXY

The NVIDIA Deep Learning Institute offers resources for diverse learning needs — from learning materials to self-paced and live training to educator programs — so individuals, teams, organizations, educators, and students get what they need to advance their knowledge in AI, accelerated computing, accelerated data science, graphics and simulation, and more.

Check out a list of self-paced courses or browse through the list of instructor-led workshops available to enterprise companies.