The influence of judges' knowledge on standard setting for competency tests was studied with 17 judges who took an economics teacher certification test while setting competency standards using the Angoff procedure. Judges tended to set higher standards for items they answered correctly and lower standards for items they answered incorrectly. (SLD)

This study examines the performance of a new method for assessing and characterizing dimensionality in test data using the NOHARM model, and comparing it with DETECT. Dimensionality assessment is carried out using two goodness-of-fit statistics that are compared to reference X[2] distributions. A Monte Carlo study is used with item parameters…

Coefficient alpha, the most commonly used estimate of internal consistency, is often considered a lower bound estimate of reliability, though the extent of its underestimation is not typically known. Many researchers are unaware that coefficient alpha is based on the essentially tau-equivalent measurement model. It is the violation of the…

A statistical test for detecting answer copying on multiple-choice tests based on Cohen's kappa is proposed. The test is free of any assumptions on the response processes of the examinees suspected of copying and having served as the source, except for the usual assumption that these processes are probabilistic. Because the asymptotic null and…

Purpose: This study evaluates the extent to which bilingual children produce the same or overlapping responses on tasks assessing semantic skills in each of their languages and whether classification analysis based on monolingual or conceptual scoring can accurately classify the semantic development of typically developing (TD) bilingual children.…

In this study, the effects of different types of support for learning from multiple representations in a simulation-based learning environment were examined. The study extends known research by examining the use of dynamic representations instead of static representations and it examines the role of the complexity of the domain and the learning…

This study investigates factors affecting the degree of novice physics students application of the naive impetus theory. Six hundred and fourteen first-year university engineering physics students answered the Force Concept Inventory as a pre-test for their calculus-based course. We examined the degree to which students consistently applied the…

A parallel design, in which items across different scales within an instrument share common stimuli and subjects respond to the common stimulus for each scale, is sometimes used in questionnaires or inventories. Because the items across scales share the same stimuli, the assumption of local item independence may not hold, thereby violating the…

Differential item functioning (DIF) in the Multidimensional Self Concept Scale (B. A. Bracken, 1992) was evaluated using 2 different methods to identify and describe DIF. Of 149 items from the Multidimensional Self Concept Scale, 42% exhibited gender DIF. Nonuniform, crossover DIF was evident in items throughout the instrument.

The standard error of measurement usefully provides confidence limits for scores in a given test, but is it possible to quantify the reliability of a test with just a single number that allows comparison of tests of different format? Reliability coefficients do not do this, being dependent on the spread of examinee attainment. Better in this…

As testing moves from paper-and-pencil administration toward computerized administration, how to present tests on a computer screen becomes an important concern. Of particular concern are tests that contain necessary information that cannot be displayed on screen all at once for an item. Ideally, the method of presentation should not interfere…

This study investigated the impact of the TI-92 handheld Computer Algebra System (CAS) on student achievement in a discrete mathematics course. Specifically, the researchers examined the differences between a CAS section and a control section of discrete mathematics on students' in-class examinations. Additionally, they analysed student approaches…

Although differential item functioning (DIF) theory traditionally focuses on the behavior of individual items in two (or a few) specific groups, in educational measurement contexts, it is often plausible to regard the set of items as a random sample from a broader category. This article presents logistic mixed models that can be used to model…

Students in four sections of an undergraduate educational course (two large and two small sections) took out-of-class practice exams prior to actual exams for each of five course units. Each course unit consisted of five class sessions focusing on a specific developmental theme. Some sections received practice-exam credit based on the number of…

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…