Discusses the test collections developed in the TREC (Text REtrieval Conference) workshops for information retrieval research and describes a study by NIST (National Institute of Standards and Technology) that verified their reliability by investigating the effect changes in the relevance assessments have on the evaluation of retrieval results.…

Describes a linkage similarity measure which takes into account both the bibliographic coupling of documents and their cocitations and compares the resulting predictions of relevance to the user's evaluations of the papers used in a test collection. (FM)

A study of relevance through comparison of texts judged by users is reported. The relevant set of documents for each information request was characterized by textual properties. The validity of procedures and interpretations is discussed. Consideration is given to possible mechanization of the process and its possible usefulness in development of…

Discusses use of associative search techniques and theoretical approaches to document retrieval problems contrasting two different ways of improving system performance: appending associative search techniques to conventional document retrieval systems and designing document retrieval systems based on probabilistic design principles. Four…

Investigates document ranking methods in thesaurus-based Boolean information retrieval systems and proposes a new thesaurus-based ranking algorithm called the Extended Relevance algorithm. Performance comparisons are made between the Extended Relevance algorithm and previous thesaurus-based ranking algorithms. (Contains 20 references.) (LRW)

Introduces the concept of cover coefficient to indexing theory and discusses a way of computing term discrimination values by using the cover coefficient concept, index-term selection, and weight modification. This method is compared to an approach that uses similarity coefficients. (Author/EM)

Presents results of simulations that were conducted to test the effects of errors in estimation of individual term probabilities on the performance of a probabilistic information retrieval (PIR) system. Assigning index terms to documents is described; an information retrieval system called Helpnet is explained; and relevance, ranking, and…

Reports research results into a methodology for determining similarity between queries characterized by Boolean search request formulations and discusses similarity measures for Boolean combinations of index terms. Rationale behind these measures is outlined, and conditions ensuring their equivalence are identified. Results of an experiment…

Discussion of the situational dynamism of user-centered relevance estimation focuses on a study that used magnitude estimation to express the relative importance of each document retrieved from an information retrieval system. Topics addressed include relevance estimation as a cognitive schema and the relationship between learning and user…

A study of logical reasoning ability of endusers searching a CD-ROM index and its correlation to searching performance was conducted at the University of Illinois at Urbana-Champaign. Findings showed a correlation between cognitive skills and citation selection and indicated that computer system design can be improved using a cognitive engineering…

Discussion of the appropriateness of evaluation criteria and measures for information retrieval focuses on a study that investigated the appropriateness of 20 measures, including precision and recall, for evaluating interactive information retrieval performance representing 4 major evaluation criteria. The importance of user satisfaction in…

Investigates relationships among: techniques used by search analysts during interviews with users before engaging in online retrieval of bibliographic citations ("open" and "closed" questions, pauses of different lengths); amount of new information gained by user as result of search; and user's satisfaction with quality of…

Demonstrates a means for estimating the number of retrieved items using well-established selective dissemination of information (SDI) profiles in the SCI, INSPEC, ISMEC, CAS, and PASCAL databases. A correlation between individual database size and number of retrieved documents in technical fields is also examined. (17 references) (LAE)

Presents brief accounts of the user-based performance measure of relevance and the information system-based performance measure of competence. Results of an experiment are reported that used a geographic information system to illustrate how collecting and analyzing data from both a system and a user view of performance can be beneficial. (73…

Compares the performance of the Target and Freestyle commercial search engines. Discusses traditional performance measures such as precision and recall, analytic models of performance, document rankings, query difficulty and correlates with other performance characteristics, and future research needs. (LRW)