Researchers from Innopolis University have come up with a way to speed up the learning of large language models by 1.5–2 times. For the first time, they used data on how a person visually perceives and reads text, the press service of the university told "The First Technical".
Modern methods of aligning AI with human preferences (RLHF) are slow and require enormous computing power. The problem is that the reward model evaluates the entire generated text with one overall score, without indicating what exactly is good or bad about it. Scientists from Innopolis proposed to look at the text through the eyes of a human.
Gaze data is collected using an eye tracker — a device mounted on a monitor. It tracks which parts of the text a person pays attention to, in what sequence, and how long they linger. Scientists conducted experiments with English texts and LLaMa and Mistral models, testing two approaches. Both gave a 1.5–2 times acceleration in learning without loss of quality.
The study proves that human gaze is a significant and previously underestimated signal in the training of AI models. Using gaze data allows you to create more efficient and economical methods for aligning language models. This not only speeds up the process, but also makes it more targeted, similar to how a teacher, instead of giving a "bad grade" for a work, points out to the student specific phrases that need to be improved. In addition, our work has shown that human gaze data can be replaced with synthetic data to solve the main problems of such studies — an acute shortage of information about gaze and the difficulty in obtaining new data.
In the future, scientists plan to test the method on offline algorithms and expand research to other languages.
Read more materials on the topic:
- A neural network will search for criminals and missing children through surveillance cameras in Rostov-on-Don
- Neural networks lie convincingly: a dangerous pattern was revealed by a study by the Perm Polytechnic
- AI will not write a new Tchaikovsky symphony: the Institute of World Economy and International Relations of the Russian Academy of Sciences explained why art remains with humans
