Tool for detecting the origin of texts written by AI created at ITMO University

Scientists at ITMO University in St. Petersburg have developed an artificial intelligence-based tool that determines with up to 94% accuracy whether a text was written by a human or an AI. The system is also capable of editing texts, reducing their "machine origin," and is already available in a demo version.

The new tool, created in ITMO's Computer Technologies Laboratory, analyzes the style and content of the text, identifying whether it was created by a human, AI, or paraphrased by AI. The algorithm successfully distinguished authorship in 94% of cases when tested on 5,500 Russian-language texts. For texts paraphrased by AI, the accuracy was 80%. The system uses two large language models that compare how "surprising" or "unexpected" the text is for them, and also analyzes linguistic features: word length, sentence structure, lexical diversity, and readability.

To train the classifier, scientists created a corpus of more than 4,000 texts in Russian, including scientific articles, essays, news, paraphrased texts, and materials generated by AI, such as ChatGPT and Gemini. An "obfuscator" was additionally developed - a tool that edits the text, eliminating traces of AI while preserving meaning and readability. It can be used to test the stability of detectors or prepare texts for publication.

A demo version of the tool is available on the Hugging Face Spaces platform, where any user can test their text. In the future, scientists plan to implement the service at ITMO to check student work and develop the project with the involvement of new researchers. The tool can be used in education, media, and business to label AI content and verify documents.

Read more materials on the topic:

ForecaState neural network framework for protecting industry from cyberattacks created at St. Petersburg Federal Research Center of the Russian Academy of Sciences

The State Duma has defined the concept of "artificial intelligence"

Helping teachers!: "Znanie" has launched a course on how to detect AI in homework assignments