Video: Smart glasses introduce AI to provide virtual assistance for people with vision impairments
Envision, which creates wearable assistive technology for people who are blind or have vision impairments, has launched a powerful new feature on its smart glasses that provides virtual assistance to the wearer about the world around them.
The Envision Glasses now incorporate a “game-changing” feature called Ask Envision, which leverages GPT-4 by OpenAI to provide virtual assistance to the wearer to better access information and navigate the environment around them.
GPT-4 is multimodal AI that accepts text and image inputs and emits text outputs. This advanced language model understands and answers natural language questions. It can be used for a wide range of applications including solving problems with great accuracy; generating, editing, and iterating with users on creative and technical writing tasks; generating captions, classifications, and analyses for images; creating long-form content; document searching; and much more.
With Ask Envision, people who are blind or have vision impairments can access any text-based content easily, eliminating the need for human assistance. The feature provides real-time information, making it easier to read, ask questions, and gather information. It further enhances independence, creating enjoyable experiences for users without having to rely on others.
Now using GPT-4, the Ask Envision feature allows Envision Glasses users to ask questions about a piece of text or document they have just scanned. This could be a menu, electricity bill, cooking instructions, and much more. The user can then ask questions about the content.
For example, after scanning a menu, the wearer could Ask Envision: “What vegetarian options are available?” Envision then tells the wearer that it is answering their question. After processing the question, the glasses could reply, for instance: “There are three vegetarian options on the menu, mushroom risotto, spinach pie, and fried tofu salad.”
Ask Envision is built into the Envision Glasses text ‘Reader’, which can be accessed after using the ‘Scan Text’ feature. Once text is processed and opened in the ‘Reader’, the user needs to press and hold the hinge button on their Envision Glasses until they hear a chime. They then simply ask their question and wait for the virtual assistant to answer.
Another example of how Ask Envision could be used is when paying an electricity bill. The wearer could ask “When is the due date?” or “How much do I need to pay?” to get the information they need.
See the Ask Envision feature in action in the video below: