We evaluate the item response theory with covariates (IRT-C) procedure for assessing differential item functioning (DIF) without preknowledge of anchor items (Tay, Newman, & Vermunt, 2011). This procedure begins with a fully constrained baseline model, and candidate items are tested for uniform and/or nonuniform DIF using the Wald statistic.…

Measurement invariance is an important prerequisite for the adequate comparison of group differences in test scores. In psychology, measurement invariance is typically investigated by means of linear factor analyses of subtest scores. These subtest scores typically result from summing the item scores. In this paper, we discuss 4 possible problems…

Differential test functioning, or DTF, occurs when one or more items in a test demonstrate differential item functioning (DIF) and the aggregate of these effects are witnessed at the test level. In many applications, DTF can be more important than DIF when the overall effects of DIF at the test level can be quantified. However, optimal statistical…

In this study, we propose that the unique needs and characteristics of linguistic minorities should be considered throughout the test development process. Unlike most measurement invariance investigations in the assessment of linguistic minorities, which typically are conducted after test administration, we propose strategies that focus on the…

Partnership for Assessment of Readiness for College and Careers (PARCC) Operational Study 4 Component 3 was designed to compare performance on PARCC mathematics field-test items for grade 3 taken with and without a drawing tool. For the 2016 testing window, five field-test items were selected to have the directions edited to allow students to…

Methods for detecting differential item functioning (DIF) and item bias are typically used in the process of item analysis when developing new measures; adapting existing measures for different populations, languages, or cultures; or more generally validating test score inferences. In 2007 in "Language Assessment Quarterly," Zumbo…

Lower reading and mathematics performance of Turkish immigrant students as compared to mainstream European students could reflect differential learning outcomes, differential socioeconomic backgrounds of the groups, differential mainstream language proficiency, and/or test bias. Using PISA reading and mathematics scores of these groups, we…

Evidence of comparability is generally needed whenever there are variations in the conditions of an assessment administration, including variations introduced by the administration of an assessment on multiple digital devices (e.g., tablet, laptop, desktop). This article is meant to provide a comprehensive examination of issues relevant to the…

The Smarter Balanced Assessment Consortium (Smarter Balanced) strives to provide every student with a positive and productive assessment experience, generating results that are a fair and accurate estimate of each student's achievement. Further, Smarter Balanced is building on a framework of accessibility for all students, including English…

In recent years, large-scale international assessments have been increasingly used to evaluate and compare the quality of education across regions and countries. However, measurement variance between different versions of these assessments often posts threats to the validity of such cross-cultural comparisons. In this study, we investigated the…

The study aims to examine whether differential item function is displayed in three different test forms that have item orders of random and sequential versions (easy-to-hard and hard-to-easy), based on Classical Test Theory (CTT) and Item Response Theory (IRT) methods and bearing item difficulty levels in mind. In the correlational research, the…

Using an empirically-based simulation study, we show that typically used methods of choosing an item calibration sample have significant impacts on achievement bias and system rankings. We examine whether recent PISA accommodations, especially for lower performing participants, can mitigate some of this bias. Our findings indicate that standard…

Assurance of learning (AoL) processes for continuous improvement and accreditation require business schools to assess program goals. Findings from the process can lead to changes in course design or curriculum. Often AoL assignments are embedded into existing courses and assessed at regular intervals. Faculty members may evaluate an assignment in…

This study examines the effectiveness of three approaches for maintaining equivalent performance standards across test forms with small samples: (1) common-item equating, (2) resetting the standard, and (3) rescaling the standard. Rescaling the standard (i.e., applying common-item equating methodology to standard setting ratings to account for…

Whereas the sample composition biases of web surveys have been discussed extensively for developed countries, studies for developing countries are scarce. This article helps to fill that gap by comparing similar non-probability-based web surveys (WEB) and probability-based face-to-face (F2F) surveys both to each other and to the labor force. An…