82516

Автор(ы): 

Автор(ов): 

3

Параметры публикации

Тип публикации: 

Доклад

Название: 

Advacheck at SemEval-2025 Task 3: Combining NER and RAG to Spot Hallucinations in LLM Answers

ISBN/ISSN: 

979-8-89176-273-2

Наименование конференции: 

  • 19th International Workshop on Semantic Evaluation (SemEval-2025)

Наименование источника: 

  • Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)

Город: 

  • Вена

Издательство: 

  • Association for Computational Linguistics Vienna, Austria

Год издания: 

2025

Страницы: 

1204-1210
Аннотация
The Mu-SHROOM competition in the SemEval-2025 Task 3 aims to tackle the problem of detecting spans with hallucinations in texts, generated by Large Language Models (LLMs). Our developed system, submitted to this task, is a joint architecture that utilises Named Entity Recognition (NER), RetrievalAugmented Generation (RAG) and LLMs to gather, compare and analyse information in the texts provided by organizers. We extract entities potentially capable of containing hallucinations with NER, aggregate relevant topics for them using RAG, then verify and provide a verdict on the extracted information using the LLMs. This approach allowed with a certain level of quality to find hallucinations not only in facts, but misspellings in names and titles, which was not always accepted by human annotators in ground truth markup. We also point out some inconsistencies within annotators spans, that perhaps affected scores of all participants.

Библиографическая ссылка: 

Вознюк А.Е., Грицай Г.М., Грабовой А.В. Advacheck at SemEval-2025 Task 3: Combining NER and RAG to Spot Hallucinations in LLM Answers / Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025). Вена: Association for Computational Linguistics Vienna, Austria, 2025. С. 1204-1210.