This study applied generalizability theory to investigate the contributions of persons, items, sections, and language backgrounds to the score dependability of the Test of English for International Communication (TOEIC). I replicated and extended Brown's (1999) study of the Test of English as a Foreign Language (TOEFL), using data from two…

In this study, mixed Rasch modeling was used on the Self-Rating Depression Scale (SDS), a widely used measure of depression, among a non-Western sample of 618 Korean college students. The results revealed three latent classes and confirmed the unidimensionality of the SDS. In addition, there was a significant effect for gender in terms of class…

This study investigates differential item functioning (DIF) in a 36-item test of English as a Second Language pragmalinguistics, assessing 254 learners' knowledge of implicature, routines, and speech acts. Two common DIF techniques, the Mantel-Haenszel method and the standardization method, detected nine items functioning differentially for test…

This paper explores alternative approaches for facilitating efficient, evidence-centered item development for a new type of verbal reasoning item developed for use on the GRE® General Test. Results obtained in two separate studies are reported. The first study documented the development and validation of a fully automated approach for locating the…

This research paper describes the benefits of using an activity-based rhetorical perspective to develop English for specific purposes (ESP) test specifications. This approach expands the potential of ESP test specifications to analyze and describe target language use (TLU) situations, TLU tasks, and ESP test tasks. Multiple activity systems are…

This study examined whether the original factor structure of the Parent Success Indicator (PSI) could be replicated with scores from generational views on both the English- and Mandarin-language versions of the instrument. The 60-item PSI was evaluated using responses from 840 Taiwanese parents (n = 429) and their 10- to 14-year-old adolescents (n…

It has been reasonably well established that test takers can sometimes answer correctly some reading comprehension passages without reading the passages on which the questions are based. This issue was studied with the new Scholastic Aptitude Test (SAT) in a study designed to determine the strategies by which examinees are able to achieve…

Recent research in cognitive psychology has drawn attention to the important role that students' personal understandings and representations of subject matter play in the learning process. This paper briefly reviews some of this research and contrasts the kind of learning that results in an individual's changed conception or view of a phenomenon…

Simulated data were used to investigate the performance of modified versions of the Mantel-Haenszel and standardization methods of differential item functioning (DIF) analysis in computer-adaptive tests (CATs). Each "examinee" received 25 items out of a 75-item pool. A three-parameter logistic item response model was assumed, and…

The difficulty of 44 items from the life sciences subscale of the National Assessment of Educational Progress (NAEP) 1985-86 science assessment was analyzed in terms of item attributes and science educators' judgments of difficulty. The attributes included ratings of various characteristics of the items' text and option set, the items' cognitive…

The development and evaluation of methods for detecting potentially biased items or differentially functioning items (DIF) represent a critical area of research for psychometricians because of the negative impact of biased items on test validity. A summary is provided of the authors' 12 years of research at the University of Massachusetts…

This evaluation package contains two model tests, one for the Alberta (Canada) French 20 course (intermediate 4/5) and one for French 30 (intermediate 5/6). The tests were designed for holistic evaluation of high school students' language performance based on an experiential/communicative approach. They allow teachers to obtain a sampling of…

This document contains test items to measure the job skills of electromechanical technicians. Questions are organized in four sections that cover the following topics: (1) shop math; (2) electricity and electronics; (3) mechanics and machining; and (4) plumbing, heating, ventilation and air conditioning, and welding skills. Questions call for…

A study was conducted to evaluate the feasibility of using item response theory (IRT) equating to reduce test form overlap of the Graduate Record Examinations (GRE) Subject Test in Mathematics. Monte Carlo methods were employed to compare double-part equating with 20-item common item blocks to triple-part equating with 10-item common item blocks.…

The Mantel-Haenszel approach for investigating differential item functioning (DIF) was applied to U.S. history items that were administered as part of the National Assessment of Educational Progress (NAEP). DIF analyses were based on the responses of 7,743 students in grade 11. On some items, Blacks, Hispanics, and females performed more poorly…