An unexpected effect has been found in compact neural networks. After a long period with almost no improvements, the model suddenly began to solve complex spatial reasoning tasks better. Researchers believe that this approach could change the creation of AI for devices where computational resources are severely limited.
This refers to recursive thinking. Instead of constantly increasing the number of neural network parameters, scientists propose making a small block of the model process the same information several times. Each subsequent pass allows for refining the solution, and alongside the intermediate result, the model stores a hidden vector – a kind of "train of thought."
Modern large language models are created with trillions of parameters. Researchers at Innopolis University tested another option: keep the model compact but give it more time to process the task.
It was here that an effect was discovered, which the researchers called "sudden learning." Rustam Lukmanov, Deputy Director of the Institute of Data Analysis and AI at Innopolis University, said that the model initially showed almost no improvement for a long time, and then demonstrated a sharp leap.
The moment of such a transition depends on the complexity of the task and the depth of recursion. This, according to the researchers, allows the model to be configured for specific requirements.
The recursive approach worked not only on abstract logical tasks. Scientists also tested it on visually grounded spatial reasoning – puzzle assembly.
The practical conclusion of the study is that the depth of a neural network's "thinking" can be increased not only by growing the model itself but also by the time it spends on reflection. At the same time, the model size can remain very small: in the experiment, it was less than a million parameters.
The article "Depth vs. Recursion: Outperforming Transformers in Puzzle Reconstruction" was published in the proceedings of one of the world's leading conferences in artificial intelligence and machine learning, ICLR 2026.
Read more on the topic:
- Innopolis scientists accelerated AI learning by modeling human gaze
- Experts Warn of Risk of Losing Independent Thinking Skills Due to AI
- Scientists at Herzen State Pedagogical University to Teach AI Empathy Using Video Games