Scientists from the St. Petersburg Federal Research Center of the Russian Academy of Sciences have developed software for controlling electronics using gestures. This was reported to RIA Novosti by the press service of the institution. The accuracy of gesture recognition is more than 99.6%, which, according to the developers, exceeds the performance of existing analogues. The system automatically recognizes 34 of the most frequently used gestures. It is enough to run the software on a laptop or computer and show the gesture to the camera. The research is supported by a grant from the Russian Science Foundation.
How the system works: three-dimensional depth map instead of regular video
The key technology is a neural network model that builds a three-dimensional depth map of the image. It allows you to determine gestures even when the background merges with the hands - the main problem of ordinary video recognition systems. The three-dimensional map adds information about the distance to each point of the image, which makes recognition resistant to complex lighting conditions and background.
"When a user wants to make a call or like a photo on a social network, it is enough for him to show a thumbs up to the camera," says Dmitry Ryumin, Senior Researcher at the Laboratory of Speech and Multimodal Interfaces of the St. Petersburg Federal Research Center of the Russian Academy of Sciences.
Application: from medicine to food production
In medicine and food production, remote control of equipment with gestures ensures a high level of hygiene - the surgeon or operator does not touch surfaces with their hands. For domestic use - managing calls, likes on social networks, selecting objects on the screen.
Specifications of the gesture recognition system of the St. Petersburg Federal Research Center of the Russian Academy of Sciences
- Recognition accuracy: more than 99.6%
- Number of recognizable gestures: 34, including the absence of a gesture
- Technology: neural network model of a three-dimensional depth map
- Equipment: standard laptop or computer camera
- Application: medicine, food production, consumer electronics
- Funding: grant from the Russian Science Foundation
Accuracy of 99.6% when using a regular camera without special depth sensors is a significant technical result. Most commercial gesture control systems require specialized cameras or sensors such as LiDAR. The development of the St. Petersburg Federal Research Center of the Russian Academy of Sciences works on standard equipment, which significantly reduces the threshold for implementation in medicine and industry.
