Very Large Scale Digital Library Building in Greenstone using Parallel Processing

Thompson, John and Bainbridge, David and Suleman, Hussein (2011) Very Large Scale Digital Library Building in Greenstone using Parallel Processing, Proceedings of 13th International Conference on Asia-Pacific Digital Libraries, 24-27 October 2011, Beijing, China, 331-340, Springer.

[img] PDF
paper.pdf

Download (133kB)

Abstract

As very large digital library collections become more commonplace, software tools must adapt appropriately. This paper reports on an evolution of the Greenstone Digital Library software to support parallel processing during the collection building phase. A series of experiments were conducted to first establish a basic speed-up factor, and then deconstruct the parallelisation process to understand the execution profile of the application. Several bottlenecks were identified and resolved to further improve the performance. The adaptation of Greenstone confirms that the build phase is indeed a suitable candidate for parallelisation; and suggests that parallelisation of processing is a new avenue for exploration in emerging digital library architectures.

Item Type: Conference paper
Uncontrolled Keywords: Greenstone, VLDL, Parallel Processing, Open MPI
Subjects: Information systems > Information systems applications
Computer systems organization > Dependable and fault-tolerant systems and networks
Alternate Locations: http://www.springerlink.com/content/06872m3x53328vr5/
Date Deposited: 09 Dec 2011
Last Modified: 10 Oct 2019 15:33
URI: http://pubs.cs.uct.ac.za/id/eprint/742

Actions (login required)

View Item View Item