Specialists from Sberbank's Center for Practical Artificial Intelligence and researchers from MIPT have proposed a new method that reduces the time and costs of training distributed neural network models.
The developed approach reduces the load on computing resources by optimizing data exchange between devices. In distributed systems, where model training takes place on multiple machines, a significant amount of time is spent on synchronization. The new method uses compression of transmitted information and takes into account the similarity of local data, which allows for less frequent data exchange without losing model quality.
Our goal was to combine modern approaches to effective communications — acceleration, compression, and consideration of data similarity — into a single algorithm with clear theoretical guarantees.
The technology is particularly relevant for tasks with limited network bandwidth, for example, when working with edge devices. According to the developers, the method can be applied in various industries, including finance, telecommunications, and industry.
The research was published as part of the AAAI’25 conference, one of the largest in the field of artificial intelligence. Further work will focus on adapting the method for various types of neural network architectures.
Thanks to such developments, voice assistants, facial recognition systems, automatic translators, and other AI services will become more efficient and accessible.
Read more materials on the topic:
Drones will be taught to search for missing people using neural networks
