Nvidia Maxine aims to fix common problems with video conferences

Nvidia’s new cloud-based solution for developers helps add AI-powered features such as face re-lighting, real-time translation, face alignment, and even gaze correction.

NVIDIA Maxine, a cloud-AI video-streaming platform, uses GANs to optimise bandwidth performance. (Nvidia)

Nvidia has introduced a new platform that can significantly enhance the videoconferencing experience for users. Called Nvidia Maxine, the platform gives developers access to a cloud-native solution that is powered by Artificial Intelligence. The solution allows developers to enable AI effects to add face alignment, gaze correction, improved resolution, and even noise cancelling during a video call.

Nvidia pointed out that its researchers chose to use a neural network instead of the software called a video codec that is generally used to compress and decompress video over the web. Nvidia said that their solution could also help reduce the bandwidth load -- down to one-tenth of H.264 using AI video compression. The Maxine platform is also cloud-based that makes it easier to deploy the solution for everyone.

“Maxine harnesses the power of NVIDIA GPUs with Tensor Cores running software such as NVIDIA Jarvis, an SDK for conversational AI that delivers a suite of speech and text capabilities. Together, they deliver AI capabilities that are useful today and serve as building blocks for tomorrow's video products and services,” Nvidia said in a post.

The company said that developers can use its platform to identify key facial points of each person on a video call and then use these points with a still image to recreate the person's face on the other side of the call. The process is done through generative adversarial networks (GANs). Then, these key points can be used for proper alignment, even when faces are rotated. A gaze correct also helps give better eye contact in case the camera is not set up properly during a call.

Another highlight of the solution is to enhance the resolution of video calls in real-time, subsequently reducing bandwidth load.

“With AI-based video compression technology running on NVIDIA GPUs, developers can reduce bandwidth use down to one-tenth of the bandwidth needed for the H.264 video compression standard. This cuts costs for providers and delivers a smoother video conferencing experience for end-users, who can enjoy more AI-powered services while streaming less data on their computers, tablets, and phones,” Nvidia added.