82485

Автор(ы): 

Автор(ов): 

3

Параметры публикации

Тип публикации: 

Статья в журнале/сборнике

Название: 

Stack More LLM’s: Efficient Detection of Machine-Generated Texts via Perplexity Approximation

ISBN/ISSN: 

1064-5624

DOI: 

10.1134/s1064562424602075

Наименование источника: 

  • Doklady Mathematics

Обозначение и номер тома: 

Т. 110, № S1

Город: 

  • New York

Издательство: 

  • Pleiades Publishing Ltd

Год издания: 

2024

Страницы: 

S203-S211
Аннотация
The development of large language models (LLMs) is currently receiving a great amount of interest, but an update of text generation methods should entail a continuous update of methods for detecting machine-generated texts. Earlier, it has been highlighted that values of perplexity and log-probability are able to capture a measure of the difference between artificial and human-written texts. Using this observation, we define a new criterion based on these two values to judge whether a passage is generated from a given LLM. In this paper, we propose a novel efficient method that enables the detection of machine-generated fragments using an approximation of the LLM perplexity value based on pre-collected statistical language models. Approximation lends a hand in achieving high performance and quality metrics also on fragments from weights-closed LLMs. A large number of pre-collected statistical dictionaries results in an increased generalisation ability and the possibility to cover text sequences from the wild. Such approach is easy to update by only adding a new dictionary with latest model text outputs. The presented method has a high performance and achieves quality with an average of 94% recall in detecting generated fragments among texts from various open-source LLMs. In addition, the method is able to perform in milliseconds, which outperforms state-of-the-art models by a factor of thousands.

Библиографическая ссылка: 

Грицай Г.М., Хабутдинов И.А., Грабовой А.В. Stack More LLM’s: Efficient Detection of Machine-Generated Texts via Perplexity Approximation / Doklady Mathematics. New York: Pleiades Publishing Ltd, 2024. Т. 110, № S1. С. S203-S211.