Multi-Hall-SA: A Cross-lingual Benchmark for Multi-Type Hallucination Detection in Low-Resource South African Languages

Ralethe, Sello and Buys, Jan (2026) Multi-Hall-SA: A Cross-lingual Benchmark for Multi-Type Hallucination Detection in Low-Resource South African Languages, Proceedings of e 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 24-29 March 2026, Rabat, Morocco, Findings of the Association for Computational Linguistics, 6282-6296, Association for Computational Linguistics.

Text 2026.findings-eacl.330.pdf - Published Version Download (270kB)

Abstract

Hallucinations generated by Large Language Models (LLMs) pose significant challenges for their application to low-resource languages. We present Multi-Hall-SA, a cross-lingual benchmark for hallucination detection spanning English and four low-resource South African languages: isiZulu, isiXhosa, Sepedi, and Sesotho. Derived from government texts, this benchmark categorizes hallucinations into four types aligned with established taxonomies of factual errors: temporal shifts, entity errors, numerical inaccuracies, and location mistakes. Human validation confirms the quality and cross-lingual alignment of our synthetically generated hallucinations. Our cross-lingual alignment methodology enables direct performance comparison between high-resource and low-resource languages, revealing notable gaps in detection capabilities. Evaluation across four state-of-the-art models shows they detect up to 23.6% fewer hallucinations in South African languages compared to English. Knowledge augmentation reduces this disparity, decreasing cross-lingual performance gaps by 59.4% on average. Beyond introducing a validated resource for low-resource languages, Multi-Hall-SA provides a framework for evaluating and improving factual reliability across linguistic boundaries, advancing more inclusive and equitable AI development. We release the dataset.

Item Type:	Conference paper
Subjects:	Computing methodologies > Artificial intelligence > Natural language processing
Alternate Locations:	https://aclanthology.org/2026.findings-eacl.330/
Date Deposited:	08 Oct 2026 11:50
Last Modified:	08 Oct 2026 11:50
URI:	https://pubs.cs.uct.ac.za/id/eprint/1804

Actions (login required)

View Item