Yandex presented to the Russian government a concept for a special set of tasks and reference data, which is proposed to be used to check AI systems for compliance with spiritual and moral values. The project is currently at the concept stage. It is unknown whether such a test will be open or closed, who exactly will determine the questions, and what the final format of the check will be.

The purpose of the test for neural networks

The creation of a national dataset – a set of tasks and correct answers for comparing different AI models – became known at a meeting of the working group on artificial intelligence regulation on August 13, 2026. According to Kommersant's sources familiar with the discussion, such a test should show how much AI corresponds to the "spirit of the times and the values of the country" and is suitable for use in Russian conditions.

A typical benchmark is a set of control tasks with reference answers, which can be used to compare different neural networks according to the same criteria. Modern tests can check not only knowledge but also the ability of AI to perform real work tasks.

Will the test be open or closed?

One of the main questions was who and how would be able to see the tasks. The FSB advocates for a closed version, Kommersant claims. If developers know the questions in advance, they will be able to specially configure their models to successfully pass the check, experts believe.

Business representatives proposed a combined option as a compromise: use a public set of tasks for open testing, and conduct the final check using a closed benchmark. The topics may coincide, but the wording of the questions will be different.

The idea is that this will allow developers to test their models in laboratories in advance, and conduct the final assessment separately and objectively.

Who will decide what determines compliance with requirements?

There is also no final answer yet to the question of who will determine the content of such a test. The IT community proposes creating a joint commission with the participation of representatives of the state, business, and expert organizations. According to the participants in the discussion, this will make the benchmark more balanced.

The office of Deputy Prime Minister Dmitry Grigorenko reported that proposals are still being discussed with industry representatives and interested departments, so it is too early to talk about specific decisions. The Ministry of Digital Development also stated that by-laws to the AI law are still being discussed with businesses. The main task is to take into account the interests of the industry and users and at the same time ensure that technologies comply with legislation. Yandex and the AI Alliance did not respond to Kommersant's request.

Experts consider closed reference answers important for testing security. If the entire test is available to developers, over time it can be learned how to pass it not by actually improving the model, but by specially configuring it for specific tasks.

But a completely closed format also creates a problem: the developer does not know exactly which categories are being checked, in what ratio, and what requirements are being imposed. In such a case, passing the test can turn into a kind of lottery, the publication writes.

There is another difficulty. For mathematics or programming, a clear correct answer can usually be given. But checking the compliance of AI with certain values is much more difficult: this will require expert evaluation and special criteria.