Intelligent AI to enhance health product accessibility for people who are blind or partially sighted
Technology giant Microsoft and Haleon, a multinational healthcare company, are working together to make health products more accessible for people with vision impairments via AI technology that narrates product labels.
Together, the companies are expanding functionality in the Microsoft Seeing AI app to give users more detailed labelling information for over 1,500 Haleon products across the US and UK. This includes for products from Sensodyne, ChapStick, and Aquafresh.
Seeing AI is a free mobile app designed to help people who have trouble seeing by narrating the world around them. In the UK, there are more than two million people with sight loss, with 340,000 people registered as blind or partially sighted.
Users can hear packaging information through the app by scanning the barcode of Haleon products. This will provide an audio read-out of important information, such as product name, ingredients, and usage instructions.
Through Seeing AI’s enhanced functionality, Haleon will help empower people to care for their own health independently by listening to label information narrated through the app.
Katie Williams, US Chief Marketing Officer at Haleon, commented: “We believe everyone should have access to self-care products, services and the information needed to make informed, proactive choices about their health needs.
“Haleon initiated this collaboration with Microsoft via its Seeing AI app to make consumer health more accessible, achievable and inclusive. We are proud to help make better everyday health more in reach for the blind and those with low vision.”
The partnership between Microsoft and Haleon was announced to mark World Sight Day on 13 October 2022.
“We have had great feedback from people who are blind and have low vision,” added Tamara Rogers, Global Chief Marketing Officer at Haleon. “They really value the increased independence that comes from being empowered to access our product information themselves.”
As well as people with vision impairments, the app supports individuals with literacy challenges.
The app was developed by a team of Microsoft engineers spearheaded by the project lead and engineering manager, Saqib Shaikh, who lost his sight at the age of seven and was driven to develop the app by his passion for using technology to improve people’s lives.
Saqib said: “I’m really excited to see the launch of this enhanced product recognition functionality, developed in collaboration with Haleon.
“Seeing AI’s intelligent barcode scanner plays audio cues to help you find the barcode, and now the information displayed for Haleon products is coming straight from the manufacturer, providing richer information including usage instructions and ingredients. This can be invaluable for someone who cannot read the label, leading to greater independence.”
Seeing AI is free to download from the Apple App Store and will be available on Android in the future.
To use Seeing AI on Haleon’s products, users should hold their phone camera over the packaging barcode. The app will read out the product name and all text on the package. Users can skip ahead or move back to the relevant section they want to listen to, for example, which flavour or how to use the product.
The Haleon barcode functionality is available in the US and UK, with plans to expand globally and add additional languages in the future, Microsoft has confirmed.