AI-based early warning system to monitor deepfakes, manipulated images

Researchers are utilising artificial intelligence (AI) to develop an early warning system that can identify manipulated images, deepfake videos and disinformation online in 2020 US election.

The project is an effort to combat the rise of coordinated social media campaigns to incite violence, sew discord and threaten the integrity of democratic elections.

According to the study, published in the journal Bulletin of the Atomic Scientists, the scalable, automated system uses content-based image retrieval and applies computer vision-based techniques to root out political memes from multiple social networks.

"Memes are easy to create and even easier to share. When it comes to political memes, these can be used to help get out the vote, but they can also be used to spread inaccurate information and cause harm," said study researcher Tim Weninger, Associate Professor at the University of Notre Dame in the US.

For the findings, the research team collected more than two million images and content from various sources on Twitter and Instagram related to the 2019 general election in Indonesia.

The results of that election, in which the left-leaning, centrist incumbent garnered a majority vote over the conservative, populist candidate, sparked a wave of violent protests that left eight people dead and hundreds injured.

The study found both spontaneous and coordinated campaigns with the intent to influence the election and incite violence.

Those campaigns consisted of manipulated images exhibiting false claims and misrepresentation of incidents, logos belonging to legitimate news sources being used on fabricated news stories and memes created with the intent to provoke citizens and supporters of both parties.

While the ramifications of such campaigns were evident in the case of the Indonesian general election, the threat to democratic elections in the West already exists.

The researchers said that they are developing the system to flag manipulated content to prevent violence, and to warn journalists or election monitors of potential threats in real-time.

The system, which is in the research and development phase, would be scalable to provide users with tailored options for monitoring content.

While many challenges remain, such as determining an optimal means of scaling up data ingestion and processing for quick turnaround, the researchers said the system is currently being evaluated for the transition to operational use.

The AI-based system is not too far behind when it comes to the possibility of monitoring the 2020 general election in the US, according to the researchers.