The identification of rapid guessing is important to promote the validity of achievement test scores, particularly with low-stakes tests. Effective methods for identifying rapid guesses require reliable threshold methods that are also aligned with test taker behavior. Although several common threshold methods are based on rapid guessing response…

Published work on vocabulary assessment has grown substantially in the last 10 years, but it is still somewhat outside the mainstream of the field. There has been a recent call for those developing vocabulary tests to apply professional standards to their work, especially in validating their instruments for specified purposes before releasing them…

This document describes the following two new student engagement metrics now included on NWEA™ MAP® Growth™ reports, and provides guidance on how to interpret and use these metrics: (1) Percent of Disengaged Responses; and (2) Estimated Impact of Disengagement on RIT. These metrics will inform educators about what percentage of items from a…

Guessing, for Pólya, is an important way of getting an initial handle on a mathematical problem. An argument can be made to place guessing in any one of the first three steps of the four-step approach to problem solving as described in "How to Solve It" (Pólya 1945). It could be a part of understanding the problem, devising a plan, or…

In recent years, the computerised adaptive test (CAT) has gained popularity over conventional exams in evaluating student capabilities with desired accuracy. However, the key limitation of CAT is that it requires a large pool of pre-calibrated questions. In the absence of such a pre-calibrated question bank, offline exams with uncalibrated…

We present a logistic function of a monotonic polynomial with a lower asymptote, allowing additional flexibility beyond the three-parameter logistic model. We develop a maximum marginal likelihood based approach to estimate the item parameters. The new item response model is demonstrated on math assessment data from a state, and a computationally…

Guessing behavior is an issue discussed widely with regard to multiple choice tests. Its primary effect is on number-correct scores for examinees at lower levels of proficiency. This is a systematic error or bias, which increases observed test scores. Guessing also can inflate random error variance. Correction or adjustment for guessing formulas…

Andersen (1995, 2002) proves a theorem relating variances of parameter estimates from samples and subsamples and shows its use as an adjunct to standard statistical analyses. The authors show an application where the theorem is central to the hypothesis tested, namely, whether random guessing to multiple choice items affects their estimates in the…

It is almost universally agreed that more frequent formative assessment (homework, clicker questions, practice tests, etc.) leads to better student performance and generally better course evaluations. There is, however, only anecdotal evidence that the same would be true for more frequent summative assessment (exams). There maybe many arguments…

This study aims to examine a) whether vocabulary knowledge, captured in the Vocabulary Levels Test (VLT), is related to the performance on the five types of reading comprehension items tested in TOEFL, i.e., Guessing Vocabulary, Main Idea, Inference, Reference, and Stated Detail; and b) whether EFL learners with different levels of vocabulary…

This article analyzes latent variable models from a cognitive psychology perspective. We start by discussing work by Tuerlinckx and De Boeck (2005), who proved that a diffusion model for 2-choice response processes entails a 2-parameter logistic item response theory (IRT) model for individual differences in the response data. Following this line…

The one-parameter logistic model with ability-based guessing (1PL-AG) has been recently developed to account for effect of ability on guessing behavior in multiple-choice items. In this study, the authors developed algorithms for computerized classification testing under the 1PL-AG and conducted a series of simulations to evaluate their…

Attali (2005) recently demonstrated that Cronbach's coefficient [alpha] estimate of reliability for number-right multiple-choice tests will tend to be deflated by speededness, rather than inflated as is commonly believed and taught. Although the methods, findings, and conclusions of Attali (2005) are correct, his article may inadvertently invite a…

The normalized gain, "g", has been an important tool for the characterization of conceptual improvement in physics courses since its use in Hake's extensive study on conceptual learning in introductory physics. The normalized gain is calculated from the score on a pre-test administered before instruction and a post-test administered…

The National Council of Teachers of Mathematics (NCTM) Algebra Standard states that instructional programs at the middle grades should enable students to "represent and analyze mathematical situations and structures using algebraic symbols" (2000, p. 222). Guess and check is a powerful problem-solving strategy that can connect a conceptual…