This article evaluates the predictions of an algorithmic-level distributed associative memory model as it introduces, propagates, and resolves ambiguity, and compares it to the predictions of computational-level parallel parsing models in which ambiguous analyses are accounted separately in discrete distributions. By superposing activation…

With the popularity of online one-to-one tutoring, there are emerging concerns about the quality and effectiveness of this kind of tutoring. Although there are some evaluation methods available, they are heavily relied on manual coding by experts, which is too costly. Therefore, using machine learning to predict instruction quality automatically…

In the current higher education landscape, universities are facing expanding requirements beyond teaching and research. Evaluation methods must evolve accordingly to prevent universities from facing development dilemmas. Current mainstream evaluation methods primarily emphasize the research domain, often failing to holistically capture a…

Designs for sequential sampling procedures that adapt to cumulative information are discussed. A familiar illustration is the play-the-winner rule in which there are two treatments; after a random start, the same treatment is continued as long as each successive subject registers a success. When a failure occurs, the other treatment is used until…

One of a series of research reports examining objective principles successfully used in other fields which can lend integrity and legitimacy to evaluation, this report presents an overview of operations research (OR) as a potential source of evaluation methodology. The nature of the methods common to this discipline are summarized and the…

Evaluates the retrieval performance of an algorithm that automatically categorizes medical documents, which consists in assigning an International Code of Disease (ICD) based on well-known information retrieval techniques. Reports on experimental results that tested precision using a database of over 20,000 medical documents. (Author/LRW)

Mislevy (1984, 1985) introduced an EM algorithm for estimating the parameters of a latent distribution model that is used extensively by the National Assessment of Educational Progress. Second order asymptotic corrections are derived and applied along with more common first order asymptotic corrections to approximate the expectations required by…

This study analyzed production of a long vowel sound within Finnish words by normal or dysphonic voices, using the Self-Organizing Map, the artificial neural network algorithm of T. Kohonen which produces two-dimensional representations of speech. The method was found to be both sensitive and specific in the detection of dysphonia. (Author/JDD)

Discussion of finding relevant documents in electronic document collections focuses on an evaluation of the Kea automatic keyphrase extraction algorithm which was developed by members of the New Zealand Digital Library Project. Results are based on evaluations by human assessors of the quality and appropriateness of Kea keyphrases. (Author/LRW)

Reviews recent research into the use of hierarchic agglomerative clustering methods for document retrieval. The topics discussed include the calculation of interdocument similarities, algorithms used to implement clustering methods on large databases, validity testing of document hierarchies, appropriate search strategies, and other applications…

Discussion of evaluation methodology in information retrieval focuses on the average precision over a set of fixed recall values in an effort to evaluate the retrieval effectiveness of a search algorithm. Highlights include a review of traditional evaluation methodology with examples; and a statistical inference methodology called bootstrap.…

Reports on a survey measuring the performance of a merging algorithm used to generate the now-defunct ULM (Union List of Monographs) file for ALEPH, Israel's research library network. Discusses automatic detection and merging of duplicate bibliographic records, problems created by lack of a standard for Hebrew spelling, and methods for measuring…

This article gives a comprehensive description of a theory for efficient assessment of knowledge. The essential concept is that the knowledge state of a subject, with regard to a specified field of information, can be represented by a particular subset of problems that the subject is capable of solving. (SLD)

Performance rating systems frequently use multiple raters in order to improve the reliability of ratings. However, unless all candidates are rated by the same raters, some candidates will be at an unfair advantage or disadvantage solely because they were rated by more stringent or lenient raters. To obtain fair and accurate evaluations of…