The purpose of this study was to apply two types of Differential Item Functioning (DIF), net and global DIF, as well as the framework of Differential Step Functioning (DSF) to real testing data to investigate measurement invariance related to test language. Data from the Program for International Student Assessment (PISA)-2006 polytomously scored…

Differential item functioning (DIF) can undermine the validity of cross-lingual comparisons. While a lot of efficient statistics for detecting DIF are available, few general findings have been found to explain DIF results. The objective of the article was to study DIF sources by using a mixed method design. The design involves a quantitative phase…

In many testing programs it is assumed that the context or position in which an item is administered does not have a differential effect on examinee responses to the item. Violations of this assumption may bias item response theory estimates of item and person parameters. This study examines the potentially biasing effects of item position. A…

This study examined how much improvement was attainable with respect to accuracy of differential item functioning (DIF) measures and DIF detection rates in the Mantel-Haenszel procedure when employing focal and reference groups with notably unbalanced sample sizes where the focal group has a fixed small sample which does not satisfy the minimum…

The purpose of this study was to examine the performance of differential item functioning (DIF) assessment in the presence of a multilevel structure that often underlies data from large-scale testing programs. Analyses were conducted using logistic regression (LR), a popular, flexible, and effective tool for DIF detection. Data were simulated…

The equating process links scores from different editions of the same test. For testing programs that build nearly parallel forms to the same explicit content and statistical specifications and administer forms under the same conditions, the linkings between the forms are expected to be equatings. Score equity assessment (SEA) provides a useful…

The Comprehensive Tests of Basic Skills, Form U, is scored in two ways: number-correct and pattern. The latter makes use of the information about which particular items are answered correctly, giving more weight to the more discriminating items and making allowances for guessing. Critics have suggested that black students are penalized by pattern…

Seven schools that have re-evaluated their needs for standardized college admissions examinations were studied to explore their admissions and innovative testing policies. The schools include: (1) Bates College in Lewiston, Maine; (2) Bowdoin College in Brunswick, Maine; (3) Harvard Graduate School of Business Administration in Cambridge,…

Based on 18 months of extensive research and study, the New Mexico State Department of Education developed the Staff Accountability Plan to address the issue of teacher accountability and certification. One of the provisions of the plan (a written assessment of general and professional knowledge for initial certification) will be fulfilled by…

Differential functioning of males and females on achievement test items was studied in a sample of 110,000 students in kindergarten through grade 12. First, item parameters and ability estimates for these parameters were obtained from LOGIST computer program runs for the entire group. Predicted performance for each item for each group was…

Test-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content…

The SAT Program undertook two studies aimed at evaluating the impact of allowing students to indicate more than one ethnic/racial category. Results of this study indicated that there is little impact on DIF [differential item functioning] analyses when different definitions of ethnic/racial classifications are used compared to traditionally…

This program of research has three general thrusts. First, the relations between motivation and achievement performance were studied across children of various sociocultural backgrounds including lower- and middle- class white, black, and hispanic children. Motivational test bias was found to be strong for students of all sociocultural…

The necessity of controlling the order in which trained essay raters for a statewide writing assessment program receive student essays was studied. The underlying theoretical question concerns possible rater bias caused by raters reading long strings of essays of homogeneous quality; this problem is usually referred to as context effect or…

Since the approval of the Basic Skills Improvement Policy in 1978, the Massachusetts Department of Education has been developing tests and alternative forms for the assessment of student achievement in five basic skills content areas: reading, writing, mathematics, listening, and speaking. Because of the lack of previous research on which to draw…