Video: This innovative phone case helps make public touchscreens more accessible
The University of Michigan, in the US, has developed pioneering new hardware for smartphones that provides a workaround for inaccessible touchscreens.
It automatically makes touchscreens in public spaces like restaurant menus, ATMs, and kiosks more accessible for disabled people.
Developed at the University of Michigan, BrushLens could help people with visual impairments, tremors, and spasms to perceive, locate, and tap buttons and keys on touchscreen menus with greater ease and independence.
Users can comb through a touchscreen interface by holding a phone connected to BrushLens against a touchscreen and dragging the phone across the screen. The phone sees what is on the screen with its camera then reads the options aloud by harnessing the phone’s built-in screen readers. Users indicate their menu choice through screen readers or an enlarged, easy-to-tap button in the BrushLens app.
When given a target, BrushLens divides the screen into a grid, then guides the user’s hand toward the section of the screen containing their menu choice by saying the coordinates of both the target and device. Once those coordinates overlap, pushbuttons or auto-clickers on the underside of the phone case tap the screen for the user, depending on the model.
Find out more about the assistive technology in the video below:
“People have to be able to operate these inaccessible touch screens in the world. Our goal is to make that technology accessible to everyone.”
Chen is the first author of a paper accepted by the Association for Computing Machinery Symposium on User Interface Software and Technology in San Francisco. He works in the lab of Anhong Guo, the University of Michigan assistant professor of computer science and engineering. Anhong led the development of BrushLens with Alanson Sample, an associate professor in the same department.
Ten study participants, six with visual impairments and four with tremors or spasms, tested the hardware and app.
“As a blind person, touch screens are pretty much inaccessible to me unless I have some help or I can plug headphones into the kiosk,” commented one study participant, Sam Rau. “Somebody else has to order for you, or they have to help you out with it. I don’t want to be in a situation where I always have to rely on the kindness of others.
“I thought about myself going into a Panera Bread and being able to order from the kiosk. I could actually see myself accomplishing something that I otherwise thought impossible.”
BrushLens also worked as intended for users whose tremors or spasms cause them to make unwanted selections on touch screens. For one participant with cerebral palsy, BrushLens improved their accuracy by nearly 74 percent, according to the university.
The inventors of BrushLens recently applied for a patent with the help of Innovation Partnerships, University of Michigan’s central hub for research commercialisation.
The team hopes to bring the product to users as an affordable phone accessory. Current estimates from Chen are that the whole device would cost under $50 (£40.60).
The team plans to further streamline their design so that it easily fits in a pocket. Offloading the battery and processing to the phone, for example, could make the design cheaper and less bulky. The companion app could also be improved by allowing users to directly interface with it via voice commands, according to Chen.
Earlier this year, assistive technology firm Smartbox launched Look Lab, a software package designed to empower disabled people by supporting practice of eye gaze skills in a relaxed and enjoyable way. The software features a comprehensive collection of 38 accessible games and creative activities that will help people get started and challenge their skills as they progress.