Describes the SAPHIRE (Semantic and Probabilistic Heuristic Information Retrieval Environment) that was developed to identify optimal approaches to indexing, retrieval, and evaluation of information retrieval systems in the biomedical domain. Six evaluation studies that have been done with SAPHIRE are summarized and future challenges are…

Discussion of Web databases, or search engines for the Internet, presents a series of questions that can be used to analyze these tools based on library literature. Six search engines are analyzed; collection methods, indexing, and abstracting are compared; and a proposal for a future ideal Web database is included. (Author/LRW)

Benchmarking is one of several alternate methods of performance evaluation, which is a key aspect in the selection of database systems. The purpose of this report is to provide a performance evaluation methodology, or benchmarking framework, to assist in the design and implementation of a wide variety of benchmark experiments. The methodology,…

The Educational Resources Information Center (ERIC) system was used to identify 1,734 documents on program evaluation methodology. Attempts to further characterize these documents by subject were only partially successful because of the lack of sufficient depth of indexing and design constraints of the computer system. Attempts to characterize…

The current state of information retrieval (IR) evaluation is reviewed with criticisms directed at the available test collections and the research and evaluation methodologies used, including precision and recall rates for online searches and laboratory tests not including real users. Automatic text retrieval systems are also discussed. (32…

Discusses cross-language, multilingual, and bilingual information retrieval on the Web; evaluates retrieval effectiveness of indexing and search strategies based on test collections from CLEF (Cross-Language Evaluation Forum) in English, French, German, and Italian; and suggests and evaluates database merging strategies. Appendices include…

The emphasis of this text is that the problems of information retrieval are the intellectual ones of subject analysis and description. These problems are not easily solved by technology alone, although certain inroads are being made by knowledge-based expert systems and other computer aids to indexing. Linguistic approaches are also showing…

A comparison of the effectiveness of information retrieval based on full-text documents with retrieval based on paragraphs, abstracts, or controlled vocabularies was accomplished using a subset of journal articles with nine search questions. It was found that full-text retrieval achieved significantly higher recall and lower precision than did the…

Describes SAPHIRE, an experimental information retrieval system featuring concept-based automated indexing, natural language input, and relevance-based retrieval; and compares its retrieval effectiveness with other indexing approaches in a Boolean-based searching environment. Indexing in MEDLINE is described, and a test collection of AIDS…

Examines the use of various statistical techniques for generating domain-oriented multiword vocabulary terms for natural language database systems. Conclusions show the vocabulary clustering effect should be considered when making significance calculations and that a simple ratio of subject matter relative frequency to total sample relative…