Generic Overgeneralization in Pre-trained Language Models

Ralethe, Sello and Buys, Jan (2022) Generic Overgeneralization in Pre-trained Language Models, Proceedings of International Conference on Computational Linguistics, 12-17 October 2022, Republic of Korea, International Committee on Computational Linguistics.

Text Generic_Overgeneralization_in_PLMs_preprint.pdf - Accepted Version Download (120kB)

Abstract

Generic statements such as “ducks lay eggs” make claims about kinds, e.g., ducks as a category. The generic overgeneralization effect refers to the inclination to accept false universal generalizations such as “all ducks lay eggs” or “all lions have manes” as true. In this paper, we investigate the generic overgeneralization effect in pre-trained language models experimentally. We show that pre-trained language models suffer from overgeneralization and tend to treat quantified generic statements such as “all ducks lay eggs” as if they were true generics. Furthermore, we demonstrate how knowledge embedding methods can lessen this effect by injecting factual knowledge about kinds into pre-trained language models. To this end, we source factual knowledge about two types of generics, minority characteristic generics and majority characteristic generics, and inject this knowledge using a knowledge embedding model. Our results show that knowledge injection reduces, but does not eliminate, generic overgeneralization, and that majority characteristic generics of kinds are more susceptible to overgeneralization bias.

Item Type:	Conference paper
Subjects:	Computing methodologies > Artificial intelligence > Natural language processing
Date Deposited:	12 Sep 2022 10:54
Last Modified:	12 Sep 2022 10:54
URI:	https://pubs.cs.uct.ac.za/id/eprint/1530

Actions (login required)

View Item