Assessment and institutional research offices have too much data and too little time. Standard reporting often crowds out opportunities for innovative research. Fortunately, advancements in data science now offer a clear solution. It is equal parts technique and philosophy. The first and easiest step is to modernize data work. This column…

With the popularity of online one-to-one tutoring, there are emerging concerns about the quality and effectiveness of this kind of tutoring. Although there are some evaluation methods available, they are heavily relied on manual coding by experts, which is too costly. Therefore, using machine learning to predict instruction quality automatically…

Although Machine Learning (ML) has already become part of our daily lives, few are familiar with this technology. Thus, in order to help students to understand ML, its potential, and limitations and to empower them to become creators of intelligent solutions, diverse courses for teaching ML in K-12 have emerged. Yet, a question less considered is…

For the last century, the quality of large-scale assessment in the United States has been undermined by narrow educational theory and hindered by limitations in technology. As a result, poor assessment practices have encouraged low-level instructional practices that disparately affect students from the most disadvantaged communities and schools.…

Recent advances in cognitive science and psychometrics have expanded the possibilities for the next generation of literacy assessment as an integrated domain (Bennett, 2011a; Deane, Sabatini, & O'Reilly, 2011; Leighton & Gierl, 2011; Sabatini, Albro, & O'Reilly, 2012). In this paper, we discuss four key areas supporting innovations in…

The present paper proposes and evaluates a readability assessment method designed for Japanese learners of EFL (English as a foreign language). The proposed readability assessment method is constructed by a regression algorithm using a new set of linguistic features that were employed separately in previous studies. The results showed that the…

The continuously growth of learning resources available in on-line repositories has raised the concern for the development of automated methods for quality assessment. The current existence of on-line evaluations in such repositories has opened the possibility of searching for statistical profiles of highly-rated resources that can be used as…

Automatic Essay Assessor (AEA) is a system that utilizes information retrieval techniques such as Latent Semantic Analysis (LSA), Probabilistic Latent Semantic Analysis (PLSA), and Latent Dirichlet Allocation (LDA) for automatic essay grading. The system uses learning materials and relatively few teacher-graded essays for calibrating the scoring…

As automated scoring of complex constructed-response examinations reaches operational status, the process of evaluating the quality of resultant scores, particularly in contrast to scores of expert human graders, becomes as complex as the data itself. Using a vignette from the Architectural Registration Examination (ARE), this article explores the…

Evaluates the retrieval performance of an algorithm that automatically categorizes medical documents, which consists in assigning an International Code of Disease (ICD) based on well-known information retrieval techniques. Reports on experimental results that tested precision using a database of over 20,000 medical documents. (Author/LRW)

Discusses how librarians can evaluate information obtained from the World Wide Web. Sources of reviews are described, explanations for 22 evaluation criteria along with an example are included, and the future of Web evaluation is considered. (LRW)

Systems that automatically assess student programming assignments have been designed and used for over forty years. Systems that objectively test and mark student programming work were developed simultaneously with programming assessment in the computer science curriculum. This article reviews a number of influential automatic assessment systems,…

This report addresses the critical importance of measuring and evaluating information systems functioning within computer network environments. The focus is on the significant benefits associated with using automated monitoring as the primary information system measurement mechanism within such computing architectures. The large scale,…

Discussion of Internet software and its use in libraries focuses on a software evaluation program at the undergraduate library at Southern Illinois University. Highlights include staff training for information technology, availability and selection of shareware and freeware, and an evaluation form for Internet software. (LRW)

This report describes the vision of the Information Systems Advisory Committee (ISAC) of an Information Systems Model for the 1990s. It includes an evaluation of the present automation environment at the university, a vision of library automation at the University of Saskatchewan between 1994 and 1999, and specific recommendations on such issues…