Scientists at the Plekhanov Russian University of Economics have found a way to significantly increase the accuracy of neural networks that predict complex processes – from atmospheric behavior to nerve cell activity. To do this, the researchers did something unusual: instead of complicating the system, they brought order to its internal structure.
This refers to reservoir neural networks. Typically, their internal layer consists of a large number of artificially connected neurons in a random manner. Due to this "chaos," the result can heavily depend on the specific random network that was formed: one works well, while another, with the same settings, performs significantly worse.
The team at Plekhanov REU replaced the random structure with small, ordered blocks – groups of three, four, or five neurons – and individual isolated nodes. The architecture with blocks of four elements showed the best results: the prediction error was reduced by up to four times, and approximately 30 times fewer connected elements were required to achieve the desired accuracy.
The development was tested on several tasks: predicting nerve cell activity, models of atmospheric dynamics and chemical reactions, and restoring hidden electroencephalogram signals. Moreover, the new system proved to be more stable – its quality was less dependent on the random initial configuration.
In the future, this approach may allow for the creation of more compact and energy-efficient neural network devices. The authors expect to apply the architecture when working with medical signals, large arrays of sensors, in energy, and even in portable neurointerfaces.
The research was led by Doctor of Physical and Mathematical Sciences Alexander Khramov, Director of the Research Institute of Applied Artificial Intelligence and Digital Solutions at Plekhanov REU. The work was supported by the Russian Science Foundation and published in the scientific journal Chaos.