VIDEO: Android accessibility updates enable people with severe motor and speech disabilities to communicate easily
To make Android more accessible for everyone, Google has introduced two new tools that make it easier for people with severe motor and speech disabilities to control their phone and communicate using facial gestures: Camera Switches and Project Activate.
Built with feedback from people who use alternative communication technology, both of these tools use an Android phone’s front-facing camera and machine learning technology to detect users’ face and eye gestures.
Camera Switches
In 2015, the technology giant launched Switch Access for Android, which lets people with limited dexterity navigate their devices more easily using adaptive buttons called physical switches.
Camera Switches, a new feature in Switch Access, turns a user’s phone camera into a new type of switch that detects facial gestures. Now, it is possible for anyone to use eye movements and facial gestures to navigate their phone hands- and voice-free.
Users can choose from one of six gestures — look right, look left, look up, smile, raise eyebrows or open your mouth — to scan and select on their phone. There are different scanning methods available, making it easy for disabled people to select their preferred option.
Users can also assign gestures to open notifications, jump back to the home screen or pause gesture detection. Camera Switches can be used in tandem with physical switches.
Google said that it created these new updates in line with feedback from people with varying speech and motor impairments, who advised that customisation options are critical. With Camera Switches, a user or a carer can select how long to hold a gesture and how big it has to be to be detected. People can use the test screen to confirm what works best for them.
To use Camera Switches, Android users should go to the Android Accessibility settings on their phone under ‘Switch Access’ or download the app. This feature is “widely available”, according to Google.
Learn more about Camera Switches in the short video below
Project Activate
Project Activate, a new Android application, lets people use the same facial gestures as Camera Switches to quickly activate customised actions with a single gesture, such as speaking a preset phrase, sending a text, making a phone call or playing audio.
This accessibility feature is designed for people to express themselves regardless of their facial mobility.
To understand how face gestures could allow for communication and personal expression, Google said it worked with numerous people with motor and speech impairments and their carers.
For example, the technology firm worked with Darren Gabbert, who is an expert at using assistive technology and communicates using a speech-generating device. He uses physical switches to type letters that his computer speaks aloud. However, this method has its limitations, Google says, as it is a slow process that makes fully participating in conversations difficult.
With Project Activate, Darren has a quick and portable way to respond in the moment, using just his phone. He can answer yes or no to questions, ask for a minute to type something into his speech-generating device, or shoot a text to his wife asking her to come in from another room.
Project Activate is available in the US, UK, Canada, and Australia in English and can be downloaded from the Google Play store.
Learn more about Project Activate in the short video below