Six articles about evaluating information retrieval (IR) systems and methods are briefly described in this introduction to a special issue and some of the major problems that are encountered in evaluating interactive information retrieval are considered. (two references) (LAE)

The Educational Resources Information Center (ERIC) system was used to identify 1,734 documents on program evaluation methodology. Attempts to further characterize these documents by subject were only partially successful because of the lack of sufficient depth of indexing and design constraints of the computer system. Attempts to characterize…

Discusses ranking techniques for document retrieval and proposes an evaluation, or filtering, technique that reduces costs and memory usage by early recognition of which documents are likely to be highly ranked. Highlights include index design, ranked query evaluation, reducing the number of accumulators, inverted file structures, and experimental…

Describes the development of a method for the evaluation and comparison of interactive information retrieval systems, which are based on the concept of a simulated work-task situation and the involvement of real end users. Highlights include real and simulated information needs; relevance assessments; and the dynamic nature of information needs.…

The current state of information retrieval (IR) evaluation is reviewed with criticisms directed at the available test collections and the research and evaluation methodologies used, including precision and recall rates for online searches and laboratory tests not including real users. Automatic text retrieval systems are also discussed. (32…

Describes a technique for automatic reformulation of Boolean queries which compares favorably with feedback as employed in a SMART system. Using patron relevance judgments, prevalence measures reflecting term distribution in relevant and nonrelevant documents are derived to guide the construction of a Boolean query for a subsequent retrieval.…

In an attempt to measure on-line searching performance, test searches of one question conducted by three bibliographic searchers are analysed to determine the effects of search strategy on results. The study reviews retrieved citations for relevance, calculating precision values, and estimated recall for each search. Performance analysis reveals…

Describes a study conducted at Oregon Health Sciences University that compared the use of three retrieval systems by medical students: a Boolean system, a word-based natural language system, and a concept-based natural language system. Results showed no statistically significant differences in recall, precision, or user preferences. (Author/LRW)

Proposes a method of evaluating information retrieval systems by concentrating on individual tools (commands, their menus or graphic interface equivalents, or a move/stratagem). A user would assess the relative success of a small part of a search, and every tool used in that part would be credited with a contribution to the result. Cumulative…

This review of the literature on full-text databases provides information on search strategy, performance measurement, and the benefits and limitations of full text compared to bibliographic database searching. Various use studies and uses of full-text databases are also listed. (21 references) (LAE)

The increasing complexity of evaluating information retrieval (IR) systems is discussed in terms of contributing factors and research methods. Examples are provided from experiments with weighted searching on a front-end system, information-seeking behavior and online catalogs, and the OKAPI experimental retrieval system. (21 references) (LAE)

Based on an analysis of online search evaluation forms collected from all types of U.S. libraries, and a pilot test of a draft evaluation form in selected federal research libraries, this report presents the work of the American Library Association's (ALA's) Machine-Assisted Reference Section (MARS) Committee on Measurement and Evaluation. The…

An adaptation of a live workshop, this guide offers pertinent information and practical techniques for users to become proficient searchers and conscious evaluators of World Wide Web resources. The book does not provide comprehensive coverage of the Internet, rather it gives the exact amount and level of information needed to perform a successful…

Examines the effectiveness of online searching for embedded stories, poems, essays, and other literary works in the Magazine Index and Modern Language Association Bibliography databases. Strategies and results of 16 test searches set up to test the access capabilities of the two databases are profiled. (SW)

Two papers are included as Part Four of this report on Salton's Magical Automatic Retriever of Texts (SMART) project report. The first paper: "A Controlled Single Pass Classification Algorithm with Application to Multilevel Clustering" by D. B. Johnson and J. M. Laferente presents a single pass clustering method which compares favorably…