This report provides evidence of the substantive and structural validity of the Knowledge for Teaching Elementary Fractions Test. Field-test data were gathered with a sample of 277 elementary educators, including teachers, administrators, and instructional support personnel, in fall 2016, as part of a larger study involving a multisite…

Experience-based cues, such as perceptual fluency, have long been thought to influence metacognitive judgments (Kelley & Jacoby, 1996; Koriat, 1997). Studies found that manipulations of perceptual fluency via changes in font and volume alter Judgments of Learning (JOLs) without influencing memory performance (Rhodes & Castel, 2008, 2009).…

The Test of Early Mathematics Ability-Third Edition (TEMA-3) is a commonly used measure of early mathematics knowledge for children aged 3 years to 8 years 11 months. In spite of its wide use, research on the psychometric properties of TEMA-3 remains limited. This study applied the Rasch model to investigate the psychometric properties of TEMA-3…

In this study, we examined the potential impact of item misfit on the reported scores of an admission test from the subpopulation invariance perspective. The target population of the test consisted of 3 major subgroups with different geographic regions. We used the logistic regression function to estimate item parameters of the operational items…

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

This study examined the performance of a proposed iterative Wald approach for detecting differential item functioning (DIF) between two groups when preknowledge of anchor items is absent. The iterative approach utilizes the Wald-2 approach to identify anchor items and then iteratively tests for DIF items with the Wald-1 approach. Monte Carlo…

In the absence of clear incentives, achievement tests may be subject to the effect of slipping where item response functions have upper asymptotes below one. Slipping reduces score precision for higher latent scores and distorts test developers' understandings of item and test information. A multidimensional four-parameter normal ogive model was…

This article provides a brief introduction to the Rasch model. Motivation for using Rasch analyses is provided. Important Rasch model concepts and key aspects of result interpretation are introduced, with major points reinforced using a simulation demonstration. Concrete guidelines are provided regarding sample size and the evaluation of items.

Questionnaires that include items eliciting count responses are becoming increasingly common in psychology. This study proposes methodological techniques to overcome some of the challenges associated with analyzing multivariate item response data that exhibit zero inflation, maximum inflation, and heaping at preferred digits. The modeling…

Purpose: Position effects may influence examinees' test performances in several ways and trigger other psychometric issues, such as Differential Item Functioning (DIF) .This study aims to supply test forms in which items in the test are ordered differently, depending on their difficulty level (from easy to difficult or difficult to easy), to…

For test scores that rely on the accurate estimation of ability via an IRT model, their use and interpretation is dependent upon the assumption that the IRT model fits the data. Examinees who do not put forth full effort in answering test questions, have prior knowledge of test content, or do not approach a test with the intent of answering…

Multiple-choice items are used in large-scale assessments of mathematical achievement for secondary students in many countries. Research findings can be implemented to improve the quality of the items and hence increase the amount of information gathered about student learning from each item. One way to achieve this is to create items for which…

In the fall of 2014, the University at Buffalo created a new universal Student Evaluation of Teaching (SET). The purpose of the present study was to establish the construct validity of SET items. Rasch analyses of data from 7 semesters (N=203,194 students) revealed problems with item fit indices and threshold distances. Changes to items and…

This reflection paper disseminates the design and implementation of an English placement test for first semester students from different majors. This design was carried out within the framework of a research project in a Colombian public university. The article has a twofold purpose: to scrutinize the stages leading to a test, and to highlight the…

The fact that Turkey's achievement in large-scale international examinations is not at the expected level is associated with such factors as students, teachers, schools, curricula and assessment approaches. The literature links student-related causes mainly to their lack of higher-order thinking skills; and teacher-related causes to incompetency…