Lip-reading PC to interpret languages

In a breakthrough study, researchers from University of East Anglia (UEA) have developed a novel computer, which can distinguish between different languages.

In a breakthrough study, researchers from University of East Anglia (UEA) have developed a novel computer, which can distinguish between different languages.

The researchers hope that the discovery could have practical uses for deaf people, for law enforcement agencies, and in noisy environments.

Lead researchers Stephen Cox and Jake Newman of UEA's School of Computing Sciences developed the novel system by statistical modelling of the lip motions made by a group of 23 bilingual and trilingual speakers.

The study showed that the computer was able to identify which language was spoken by an individual speaker with very high accuracy.

These languages included English, French, German, Arabic, Mandarin, Cantonese, Italian, Polish and Russian.

"This is an exciting advance in automatic lip-reading technology and the first scientific confirmation of something we already intuitively suspected -that when people speak different languages, they use different mouth shapes in different sequences," said Prof Cox.

"For example, we found frequent 'lip-rounding' among French speakers and more prominent tongue movements among Arabic speakers," he added.

The researchers would be conducting further studies to make the system more robust to an individual's physiology and his or her way of speaking.