Large language models require a lot of memory and powerful computing accelerators. However, MIPT has developed the HARP method, which helps compress neural networks more effectively without significant loss of quality. In one experiment, the speed of an AI model after two-bit quantization was about 128 tokens per second — approximately twice as fast as the original version with 16-bit parameters. The authors tested the approach on language models of various sizes, including Qwen3-8B.

To understand how HARP works, one needs to understand why neural networks consume so much memory. When operating, a language model uses billions of parameters — numbers that determine how it processes queries and forms responses. Typically, 16 bits per parameter are used to store these values, but they can be represented with fewer bits. This process is called quantization.

The stronger the compression, the less memory the model requires. However, with too coarse a representation of numbers, the neural network may perform worse in answering questions and solving tasks. Therefore, developers have to find a balance between model size and its performance quality.

HARP helps reduce losses during such compression. Before quantization, the algorithm selects its own mathematical transformations for each layer of the neural network. These change the data representation so that values are easier to compress. At the same time, there is no need to retrain the entire language model: the adjustment is performed on a special data sample.

The authors tested the method on models ranging from 1 billion to 70 billion parameters. In experiments, HARP improved quality metrics compared to an approach that used fixed transformations. The results were also tested on Qwen3-8B, which allowed evaluating the method's performance beyond the Llama family.

In a separate experiment with a 7-billion-parameter model, the two-bit version with HARP processed about 128 tokens per second. For comparison, the original model with 16-bit parameters showed about 61 tokens per second, and a variant with another transformation method showed about 142 tokens per second. As a result, the compressed model retained approximately 90% of the speed of the second variant and worked approximately 2.1 times faster than the original FP16 version.

According to the developers, such methods can be useful for companies that need to run language models on hardware with limited memory. The fewer resources a single model requires, the more models and tasks can potentially be hosted on existing servers. However, HARP does not eliminate the need for computational power and electricity, but helps to use available equipment more efficiently.