Exploring the feasibility of ontology-based educational question generation with an LLM

Cardillo, Franco Alberto and Keet, C. Maria and Mahlaza, Zola (2026) Exploring the feasibility of ontology-based educational question generation with an LLM, Proceedings of 19th International Conference on Natural Language Generation, 17-21 October 2026, Utrecht, the Netherlands.

Text INLG2026.pdf - Published Version Download (632kB)

Abstract

Educational question generation from structured knowledge, such as represented in knowledge graphs and ontologies, are typically generated by traditional NLP approaches. They face issues of template or grammar generation and the many corner cases to take into account. LLMs are also not assumed to perform well, but then due to hallucinations that are unacceptable for educational questions, and lack of training data in specialised domains of the ontologies may affect more complex question involving more and more expressive axioms. We assessed the question generation problem with 7 ontologies on 7 relatively small LLMs, all executed on GPU-enabled machines, through prompting from 0 to 10-shots on eight different types of educational questions. The first hypothesis, on LLMs producing grammatically better quality sentences but having more hallucinations than the NLP pipeline approach, was falsified, and the larger problem was the answer appearing in the generated question. The second hypothesis, on LLMs performing better with simple subsumption axioms than with 'complex' ones, was validated. We conclude that the tested LLMs are not well-suited for reliable ontology verbalisation as question generation, compared to tailored NLP pipelines

Item Type:	Conference paper
Subjects:	Computing methodologies > Artificial intelligence > Natural language processing Information systems > Information retrieval > Document representation > Ontologies
Date Deposited:	20 Oct 2026 07:50
Last Modified:	20 Oct 2026 07:50
URI:	https://pubs.cs.uct.ac.za/id/eprint/1825

Actions (login required)

View Item