We have developed simple data-mining algorithms to assess the consistency and the randomness of student responses to problems consisting of multiple true or false statements. In this paper we describe the algorithms and use them to analyze data from introductory physics courses. We investigate statements that emerge as outliers because the class…

This study investigated the effect of targeted test preparation, or coaching, on oral English as a second language test scores. The tests in question were the Basic English Skills Test Plus (BEST Plus), a scripted oral interview published by the Center for Applied Linguistics, and the Versant English Test (VET), a computer-administered and…

Although traditional methods to collect similarity data (for multidimensional scaling [MDS]) are robust, they share a key shortcoming. Specifically, the possible pairwise comparisons in any set of objects grow rapidly as a function of set size. This leads to lengthy experimental protocols, or procedures that involve scaling stimulus subsets. We…

Indirect metaphors are pervasive in everyday language: People talk about "long" vacations, "short" tempers, and "colorful" language. But, why do we use concrete lexical items that are associated with the physical world when we talk about abstract, or non-physical, concepts? A potential answer is provided by proponents…

Most computerized adaptive tests (CAT) have been studied under the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CAT. In addition, a number of psychological variables (e.g., quality of life, depression) can be conceptualized…

In this paper we present a model for computerized adaptive practice and monitoring. This model is used in the Maths Garden, a web-based monitoring system, which includes a challenging web environment for children to practice arithmetic. Using a new item response model based on the Elo (1978) rating system and an explicit scoring rule, estimates of…

Aim: The aims of the study were to: (1) build new item banks for a revised version of the Pediatric Evaluation of Disability Inventory (PEDI) with four content domains: daily activities, mobility, social/cognitive, and responsibility; and (2) use post-hoc simulations based on the combined normative and disability calibration samples to assess the…

Objective: The study evaluated the effectiveness of an online first aid course by comparing it with the traditional instructor-led course. An effective online course increases course accessibility and mitigates the major deterrent to widespread layperson training. Design: A comparison group design evaluated performances among 25 laypersons…

The current study was designed to extend the current literature to study scale drift in CAT as part of improving quality control and calibration process for ACCUPLACER, a battery of large-scale adaptive placement tests. The study aims to evaluate item parameter drift using empirical data that span four years from the ACCUPLACER Arithmetic…

The "Boston Globe" investigation shows that cheating on US Army exams is truly massive in scale, dwarfing any runof-the-mill scandal. According to the "Boston Globe," "For eight years the Army has known that its largest online testing program--which verifies that soldiers have learned certain military skills and helps them amass promotion…

This paper discusses a possible solution to a problem frequently encountered by educators seeking to use computer-based or multiple choice-based exams for mathematics. These assessment methodologies force a discrete grading system on students and do not allow for the possibility of partial credit. The research presented in this paper investigates…

This simulation study sought to compare four different computerized adaptive testing (CAT) content-balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification, validity of clinical diagnostic recommendations, and sensitivity to atypical responding. The four…

When respondents answer paper-and-pencil (PP) questionnaires, they sometimes modify their responses to correspond to previously answered items. As a result, this response bias might artificially inflate the reliability of PP questionnaires. We compared the internal consistency of PP questionnaires to computerized questionnaires that presented a…

This study explored EFL students and faculty's perceptions of and attitudes towards the use of online assessment and practice. A descriptive method was used, employing quantitative data collection and analyses from a sample of 400 students of different age categories and educational levels and another sample of 25 teachers in the English…

Language tools are reflections of the academic traditions of assessment developers, as well as the perceptions of language quality as held by a society. The certification of Quebec's teachers offers an interesting case in point: the Québec Government's "ministère de l'Éducation, du Loisir et du Sport" (MELS) requires that all students in…