A Hybrid Distributed Architecture for Indexing

Nakashole, Ndapandula and Suleman, Hussein (2009) A Hybrid Distributed Architecture for Indexing, Proceedings of 13th International Conference on European Digital Libraries (ECDL), Corfu, Greece.

[img] PDF
indexing_ecdl.pdf

Download (255kB)

Abstract

This paper presents a hybrid scavenger grid as an underlying hardware architecture for search services within digital libraries. The hybrid scavenger grid consists of both dedicated servers and dynamic resources in the form of idle workstations to handle medium- to large-scale search engine workloads. The dedicated resources are expected to have reliable and predictable behaviour. The dynamic resources are used opportunistically without any guarantees of availability. Test results confirmed that indexing performance is directly related to the size of the hybrid grid and intranet networking does not play a major role. A system-efficiency and cost-effectiveness comparison of a grid and a multiprocessor machine showed that for workloads of modest to large sizes, the grid architecture delivers better throughput per unit cost than the multiprocessor, at a system-efficiency that is comparable to that of the multiprocessor.

Item Type: Conference paper
Uncontrolled Keywords: hybrid grid architecture, indexing
Subjects: Information systems > Information retrieval
Date Deposited: 08 Jun 2009
Last Modified: 10 Oct 2019 15:34
URI: http://pubs.cs.uct.ac.za/id/eprint/522

Actions (login required)

View Item View Item