Kivaisi, Alexander and Mbogho, Audrey (2012) Web-based corpus acquisition for Swahili language modelling, Proceedings of 3rd International Workshop on Spoken Language, 7-9 May 2012, Cape Town, South Africa, 42-47.
![[img]](http://pubs.cs.uct.ac.za/style/images/fileicons/application_pdf.png) |
PDF
sltu2012_kivaisi_mbogho.pdf Download (341kB) |
Abstract
Finding large amounts of text data for use in natural language technology is difficult for under-resourced languages such as Swahili. The corpora that are readily accessible for these languages are not sufficient to be used in language technologies, whose requirements can run into the hundreds of millions of words. This paper describes how we can take advantage of search engines such as Google together with crawling tools to collect Swahili text from the Web. We also share the experience of cleaning up and normalising the resulting text data. Finally, we show some preliminary results of the evaluation of the language models built from our corpus as well as results of how they compare to those built from the Helsinki Corpus.
| Item Type: | Conference paper |
|---|---|
| Uncontrolled Keywords: | Under-resourced languages, corpus acquisition, Swahili, language modelling |
| Subjects: | Applied computing > Document management and text processing |
| Date Deposited: | 28 Nov 2012 |
| Last Modified: | 10 Oct 2019 15:33 |
| URI: | http://pubs.cs.uct.ac.za/id/eprint/842 |
Actions (login required)
 |
View Item |