Scientists from MIPT have developed a new structure of a bipolar morphological neuron – a simplified analogue of traditional artificial neurons – and proposed a learning method that can increase the computational efficiency of neural networks.
The new method of building a neural network demonstrates high results even with a simplified neuron structure. This approach opens up opportunities for creating more energy-efficient and faster artificial intelligence systems, which is especially important given that most modern AI models rely on addition and multiplication operations, with multiplication requiring significantly more computing power.
An alternative to traditional neurons is bipolar morphological neurons, in which the multiplication operation is replaced by taking the maximum. Although neural networks based on them may have an advantage in energy efficiency, their training presented a serious challenge due to the increased number of computational branches.
Russian researchers have succeeded in creating a new model of such a neuron, which reduces the number of branches from four to one. A new learning method based on knowledge distillation – transferring information from a "teacher" network to a trained one – allows achieving accuracy comparable to classical models. Experimental tests of analogues of LeNet and ResNet-22, popular systems for image and handwriting recognition, showed that the new architecture copes with tests at the same level as its traditional analogues.
In the future, the development of such energy-efficient and high-performance models will allow creating more economical algorithms for tasks requiring significant computing resources, which could be an important step in the development of artificial intelligence technologies.
Read materials on the topic:
Skoltech and MIPT have developed an algorithm for quickly finding highly efficient alloys using AI
An AI system has been developed to detect errors in quantum computing
