Yandex has open-sourced Alice AI Search Pretrain — the language model underlying fast AI answers in Yandex Search. Anyone can download it: the company has released the model on the Hugging Face platform under the Apache 2.0 license.

Open source means open source code: developers and researchers can study the model's structure and use it in their projects. The Apache 2.0 license freely permits this, including in commercial products.

Pretrain is the basic version after the first training stage: the model already knows a lot about the world but is not yet tailored for a specific task. The fine-tuned version has been generating answers under the Yandex search bar since July.

Alice AI Search Pretrain is built on a hybrid architecture of two technologies. Encoder-Decoder — one part of the model reads and selects information, the other writes the answer, so it turns out short and to the point. MoE (Mixture of Experts) — instead of the entire neural network, only the block that is better versed in the topic is activated.

Parameters are numerical settings that store the model's "knowledge": there are 35 billion in total, but for one token (a fragment of text roughly the size of a word), about 600 million are used, less than 2%.

In terms of answer quality in Russian, the model outperformed other compact models — Qwen 3.5 2B and 4B, T5 Gemma 2 4B-4B Base. Measurements were conducted using a blind method: the evaluator did not know which model provided the answer. At the same time, Alice AI Search Pretrain is comparable in quality to Qwen 3.5 35B-A3B, which requires significantly more computational resources. More than 49 million people use fast answers in Yandex Search monthly.

Read more on the topic:

Комментарии

правилами