AI can screen online hate speech and give control back to users: New research

As online hate speech in India and elsewhere is increasingly seen as a threat to democracy, researchers are developing artificial intelligence that will, like malware filters, make 'quarantining' or controlling exposure to hate speech possible without resorting to censorship.

A linguist and an engineer at the University of Cambridge who came up with the idea to contain the spread of hate speech via social media have published their proposal in the journal Ethics and Information Technology.

The experts are using databases of threats and violent insults to build algorithms that can provide a score determining the likelihood of an online message containing forms of hate speech. As these algorithms get refined, potential hate speech could be identified and 'quarantined'.

Users could receive a warning alert with a 'Hate O'Meter' - the hate speech severity score - the sender's name, and an option to view the content or delete it unseen, akin to spam and malware.

"Hate speech is a form of intentional online harm, like malware, and can therefore be handled by means of quarantining," says co-author and linguist Stefanie Ullman. "In fact, a lot of hate speech is actually generated by software such as Twitter bots."

An example might involve a word like 'bitch': A misogynistic slur, but also a legitimate term in contexts such as dog breeding. It is the algorithmic analysis of where such a word sits syntactically - the types of surrounding words and semantic relations between them - that informs the hate speech score.

"Companies like Facebook, Twitter and Google generally respond reactively to hate speech," adds co-author and engineer Marcus Tomalin. "This may be okay for those who don't encounter it often. For others it's too little, too late."

"Many women and people from minority groups in the public eye receive anonymous hate speech for daring to have an online presence. We are seeing this deter people from entering or continuing in public life, often from groups in need of greater representation," he says.

The researchers say their proposal is not a magic bullet, but sits between the "extreme libertarian and authoritarian approaches" of either entirely permitting or prohibiting certain language online. It makes the user the arbiter.

Tomalin says: "Many people don't like the idea of an unelected corporation or micromanaging government deciding what we can and can't say to each other. Our system will flag when we should be careful, but it's always your call".

"It doesn't stop people posting or viewing what they like, but it gives much needed control to those being inundated with hate," added Tomalin.