Google Lookout ‘Image Q&A’ uses AI to offer detailed image descriptions to visually impaired

Google Lookout 'Image Q&A' feature uses an advanced visual language model developed by Google DeepMind to generate 'more detailed image descriptions'.

Google Lookout | Google Lookout Image Q&A | Google Lookout new features

Google Lookout is an app designed to help visually impaired people. (Image Source: Google)

Listen to this article

00:00

1x 1.5x 1.8x

Google Lookout, an Android app designed to help blind people and those with low vision is getting some new features. Launched in 2019, the app helps them accomplish everyday tasks like putting away groceries or sorting their mail.

In a blog post, the tech giant said Lookout’s ‘Image Q&A’ feature lets people with vision problems scroll through images on their camera app, social media or group chats. The company says it is using an advanced visual language model developed by Google DeepMind to generate ‘more detailed image descriptions.’

Users can also upload

an image and get information about it even if there are no captions or alt text. After Lookout provides a description of what’s on the screen, they can ask follow-up questions about the photo by either typing or using voice.

Story continues below this ad

Image Q&A is currently available in the UK, US and Canada in English. (Image Source: Google)

For the uninitiated, ‘Alt text’ describes what the image is about and can be used by screen readers to help visually impaired people understand what they are looking at. But the problem is that images shared on messaging apps or social media have low-quality captions and alt text, with the majority of them not having any ‘Alt text’ at all. This makes most photos inaccessible to people with visual impairment.

Also Read | Google unveils new Android logo, adds AI to ‘At a Glance’ widget

Currently, the Image Q&A feature is available to those in the United States, United Kingdom and Canada in English, but Google says it will soon be available globally. Lookout is also adding support for 11 new languages like Korean, Japanese and Chinese, bringing the total count to 34.

In the last few years, Google has made numerous announcements to improve digital accessibility. While the company’s Project Relate helps those with speech impairments communicate with others, Live Transcribe provides realme speech-to-text to text transcriptions for those who have problems hearing.

Apple has also come up with several useful features like ‘People Detection’, which lets blind people or those with low vision know how close the other person is to them. The Cupertino-based tech giant also provides Live Captions on FaceTime.