Last week, Nvidia launched a new version of Nvidia Broadcast (opens in new tab) — the deep learning and AI-powered software that can do noise suppression, background removal/replacement, camera framing, and now… Eye Contact. That last one is currently in beta, probably for good reason, and you can see what it looks like “live” in the above test video.
AI and deep learning have been in the news a lot lately, for good reason. Stuff like Dall-E, Midjourney, and Stable Diffusion are creating art from text, often with rather striking results. And other times, you end up with mangled mutant creatures and the dual head, multiple limbs problem. On the text side, ChatGPT might be sounding the death knell for English essays and journalism — and no, it did not write this news post.
The idea behind Eye Contact is simple enough to express: When you’re on a webcast or meeting, often you look away from the camera. In fact, there’s a real chance you’re almost always looking away from the camera because it’s sitting at the top of the screen, while the things you want to look at are actually on the screen. So what if you train an AI model on faces and teach it to correct images where someone isn’t looking straight into the lens? Get millions of images that are appropriately tagged, feed it into the network, and out pops an amazing tool, right?
Obviously it’s not that simple, because Nvidia has been talking about its Eye Contact feature for well over a year, and yet it’s only now going into public release. Differences between all the myriad faces around the world makes it a tough problem to “solve,” and even now the results are imperfect, if we’re putting it nicely.
One of the things I noticed in testing is that often the live video feed would oscillate between me looking at the camera and me looking elsewhere, even though my focus was generally in the same spot. I’m not sure if that’s intentional — having someone staring directly into the camera during an entire video chat would start to feel creepy — but if there’s some timing involved, it should be adjusted.
What’s more difficult to say is whether or not this sort of effect is even beneficial in the whole. If you want to look like you have eye contact with the camera, shouldn’t you just learn to look into the camera lens? Solving human error through AI might just create sloppy behavior, and then what happens if you end up on a camera that doesn’t autocorrect your feed?
But whether or not you like it, Nvidia Broadcast with Eye Contact is now available for RTX owners everywhere to try out. My testing was done with an RTX 3090 Ti, but so far the system requirements still list an RTX 2060 as the entry point, and that includes mobile RTX 3050 GPUs as well, as far as I understand things (though I don’t have access to one of those).
Long-term, I suspect at some point Nvidia will end up with certain AI models that are more complex and require faster hardware than an RTX 2060, just like DLSS 3’s Frame Generation feature requires an RTX 40-series graphics card, but for now any RTX GPU made in the past four years will suffice.
Do you like the effect, hate it, find it creepy, or something else? Let us know in the comments, along with other effects you’d rather see. I’m looking forward to the time when we can all have virtual cartoon avatars like Toy Jensen talking in place of real people, perhaps reading articles that were written by AI, with the videos and articles both being consumed by AI. It’s bots all the way down from there.
