Facebook improves its AI tool that generates photo descriptions for visually impaired users

Facebook has updated its Automatic Alt Text (AAT) system that is used to generate automated photo descriptions for people with visual impairments. Facebook said its AAT system can now identify a photo 10x more than before, and identify more details in a photo.

Facebook introduced AAT back in 2016, and it has since been improving the technology with new capabilities. Previously, the photo description would be the word “photo” and the name of the person who shared. In the latest update, Facebook’s AAT can offer more detailed descriptions such as identifying activities, landmarks, type of animals and more.

Facebook also cited an example of a photo saying, “May be a selfie of 2 people, outdoors, the Leaning Tower of Pisa.” It can also include information such as the positional location and relative size of elements in a photo. Facebook’s AAT can now say where exactly people are standing in a photo instead of just saying the number of people in the photo. It has also been improved to highlight an object that is larger in the photo as the primary object. It also added that the ATT still uses “may be” for every description as there could be errors in them.

“AAT uses simple phrasing for its default description rather than a long, flowy sentence. It’s not poetic, but it is highly functional. Our users can read and understand the description quickly — and it lends itself to translation so all the alt text descriptions are available in 45 different languages, ensuring that AAT is useful to people around the world,” Facebook explained in a blog post.

Facebook’s AAT can now recognise recognise over 1,200 concepts which is 10 times more than its original model.