Video: US engineers develop smart walking stick that helps with everyday tasks

Engineers from the University of Colorado Boulder (CU Boulder) in the US have developed an AI-powered walking stick that can help people with vision impairments or those who are blind navigate daily tasks easily, such as shopping for a box of cereal or finding a seat in a busy restaurant.

The assistive technology incorporates additional features to a standard walking stick. Using a camera and computer vision technology, the walking stick maps and details the world around it. It then guides users by using vibrations in the handle and spoken directions, such as “reach a little bit to your right.”

Shivendra Agrawal, a doctoral student in the Department of Computer Science at CU Boulder, said: “I really enjoy grocery shopping and spend a significant amount of time in the store. A lot of people can’t do that, however, and it can be really restrictive. We think this is a solvable problem.”

In a study published in October 2022, Shivendra and his colleagues in the Collaborative Artificial Intelligence and Robotics Lab got one step closer to solving this issue.

According to Shivendra, the walking stick prototype is not supposed to be a substitute for designing places like grocery stores to be more accessible. Instead, it is supposed to showcase the power of AI to help people become more independent.

Shivendra and his colleagues first explored AI’s potential by tackling a familiar problem: “Where do I sit?”

“Imagine you’re in a café,” he said. “You don’t want to sit just anywhere. You usually take a seat close to the walls to preserve your privacy, and you usually don’t like to sit face-to-face with a stranger.”

To see if their smart walking stick could facilitate choosing a specific seat, CU Boulder researchers set up a café in their lab, complete with several chairs, patrons, and a few obstacles.

Study subjects strapped on a backpack with a laptop in it and picked up the smart walking stick. They swivelled to survey the room with a camera attached near the cane handle. Like a self-driving car, algorithms running inside the laptop identified the various features in the room and then calculated the route to an ideal seat.

The study showed promising results; subjects were able to find the right chair in 10 out of 12 trials with varying levels of difficulty. So far, the subjects have all been sighted people wearing blindfolds.

However, the researchers plan to evaluate and improve their assistive device by working with people who are blind or visually impaired once the technology is more dependable.

Another area that the CU Boulder research team has explored is whether the smart cane can assist with grocery shopping. This new research has not yet been published.

Shivendra and his colleagues adapted their device to be able to find and grasp products in aisles filled with dozens of similar-looking and similar-feeling choices.

Again, the team set up a makeshift environment in their lab: a grocery shelf stocked with several different kinds of cereal. The researchers created a database of product photos into their software. Study participants then used the walking stick to scan the shelf, searching for the product they wanted.

“It assigns a score to the objects present, selecting what is the most likely product,” Shivendra explained. “Then the system issues commands like ‘move a little bit to your left.’”

He added that it will be a while before the team’s walking stick makes it into the hands of real shoppers. The group, for example, wants to make the system more compact, designing it so it can run off a standard smartphone attached to a cane.

“Our aim is to make this technology mature but also attract other researchers into this field of assistive robotics,” concluded Shivendra. “We think assistive robotics has the potential to change the world.”

Watch the smart walking stick in action in the short video below: