Indian-origin scientists develop AI system to detect deepfakes

The AI system can currently identify deepfake still images but it will soon help detect videos as well.

Deepfake videos pose a potential threat. (Getty Images/iStockphoto)

At a time when deepfake videos become a new threat to users' privacy, a team of Indian-origin researchers has developed Artificial Intelligence (AI)-driven deep neural network that can identify manipulated images at the pixel level with high precision.

Realistic videos that map the facial expressions of one person onto those of another -- known as deepfakes, present a formidable political weapon in the hands of nation-state bad actors.

Led by Amit Roy-Chowdhury, professor of electrical and computer engineering at the University of California, Riverside, the team is currently working on still images but this can help them detect "deepfake" videos.

"We trained the system to distinguish between manipulated and nonmanipulated images and now if you give it a new image, it is able to provide a probability that that image is manipulated or not, and to localize the region of the image where the manipulation occurred," said Roy-Chowdhury.

A deep neural network is what AI researchers call computer systems that have been trained to do specific tasks, in this case, recognize altered images. These networks are organized in connected layers; "architecture" refers to the number of layers and structure of the connections between them.

While this might fool the naked eye, when examined pixel by pixel, the boundaries of the inserted object are different.

For example, they are often smoother than the natural objects. By detecting boundaries of inserted and removed objects, a computer should be able to identify altered images. The researchers tested the neural network with a set of images it had never seen before, and it detected the altered ones most of the time. It even spotted the manipulated region.

"If you can understand the characteristics in a still image, in a video it's basically just putting still images together one after another," explained Roy-Chowdhury in a paper published in the journal IEEE Transactions on Image Processing.

"The more fundamental challenge is probably figuring out whether a frame in a video is manipulated or not".

Even a single manipulated frame would raise a red flag.

Roy-Chowdhury, however, thinks we still have a long way to go before automated tools can detect "deepfake" videos in the wild.

"This is kind of a cat and mouse game. This whole area of cybersecurity is in some ways trying to find better defense mechanisms, but then the attacker also finds better mechanisms."