The first intelligent assistant for visually impaired and blind people has been developed in Novosibirsk. The device is based on artificial intelligence technologies and large language models.
The project team includes specialists from the NSU AI Center and the branch of the Intersectoral Research and Technology Complex "Eye Microsurgery". Their goal is to create a compact wearable device that will help people navigate in space using sound.
The system works as follows: a video camera captures the surrounding environment, the image is transmitted to a portable computer, which converts it into text. Then the text is voiced by a voice assistant and transmitted to the user through headphones.
The device can already recognize inscriptions, point out obstacles, and describe objects and rooms. In the future, it is planned to add the ability to ask questions about objects and receive detailed answers.
The development is based on a multimodal language model. Scientists believe that this could become an affordable alternative to expensive medical implants.
The prototype of the device is planned to be completed by the end of 2025. It will include a 16-core single-board computer, a video camera, and headphones, and the total weight of the system will not exceed one kilogram.
Read more on the topic:
AI for teaching the blind developed in Novosibirsk
Microstent for combating glaucoma developed at the Research Institute of Eye Diseases