Uses a reliability framework to examine the role that relevance judgments play in quantitative information retrieval (IR) performance measurement with an eye toward issues introduced by Internet IR. Argues that a theoretical basis for reliable measurement is possible if the relevance yardstick is reconsidered in light of recent research and if the…

Describes computer program which converts text of user input and system responses from online search system into fixed format records describing interaction. Query language syntax, format of output record produced by parser, problems in constructing parser, its performance characteristics, and recommendations for improving process of logging…

The Comparative Systems Laboratory (CSL) proposed to deal with the problems of testing retrieval systems by examining a number of processes involved in such systems, with particular attention to the human factor. It is hoped that the results of the project will be useful in refining methodologies for experimentation with information retrieval (IR)…

Describes the comparative approach to system evaluation used in a project which administered an online retrieval test to an experimental online catalog to produce data for evaluating effectiveness of a new subject access design. Discusses efforts used to ensure data reliability, and strategies the researchers can use to improve on the data…

The first task of the study was to develop valid, short-cut methods of indexing the narrative content of evaluation reports that would extract the differentiating information contained in evaluative comments simply, reliably, and with as good classification accuracy as the longer initial procedure. In the second task, the original inter-indexer…

The IADIS CELDA 2012 Conference intention was to address the main issues concerned with evolving learning processes and supporting pedagogies and applications in the digital age. There had been advances in both cognitive psychology and computing that have affected the educational arena. The convergence of these two disciplines is increasing at a…