Over the past decades, the application of Rasch measurement in language assessment has gradually increased. In the present study, we coded 215 papers using Rasch measurement published in 21 applied linguistics journals for multiple features. We found that seven Rasch models and 23 software packages were adopted in these papers, with many-facet…

This article presents the author's response to Xiaoming Xi's paper titled "How do we go about investigating test fairness?" In the paper, Xi offers "a means to fully integrate fairness investigations and practice". Given the current importance accorded to fairness in the language testing community, Xi makes a case for viewing fairness as an aspect…

Advances in natural language processing (NLP) and automatic speech recognition and processing technologies offer new opportunities for language testing. Despite their potential uses on a range of language test item types, relatively little work has been done in this area, and it is therefore not well understood by test developers, researchers or…

Based on the observation that scale-level methods are sometimes exclusively used to investigate measurement invariance for test translation, describes results of a simulation study investigating whether item-level differential item functioning (DIF) manifests itself in scale-level analyses such as single and multigroup factor analyses and per…

Investigates differential item functioning (DIF) across two different broad language groupings, Asian and European, in a speaking test in which the test takers' responses were rated polytomously. Data were collected from 1038 nonnative speakers of English from France, Hong Kong, Japan, Spain, Switzerland, and Thailand who took the SPEAK test in…

Reports on the potential problems in scoring responses to sequencing tests, the development of a computer program to overcome these difficulties, and an exploration of the value of scoring procedures. (Author/VWL)

Explored the relative contributions to Test of English as a Foreign Language (TOEFL) score dependability of various numbers of persons, items, subtests, languages, and their various interactions. Sampled 15,000 test takers, 1000 each from 15 different language backgrounds. (Author/VWL)

Describes a statistical method for evaluating the translation equivalence of language test items that are scored dichotomously. Provides an illustration of the method to a portion of the verbal subtest of the Psychometric Entrance Test, which is a large-scale postsecondary admissions test used in Israel. (VWL)

Analyzes gender-uniform differential item functioning (DIF) in a second language vocabulary test with the tools of item response theory to study potential gender impact on the test performance measured by different item composites. Results show that while there are test items with indications of DIF in favor of either females or males, the test as…

A nested cross-over design was used to examine the effects of visual condition, type of stimuli, and language proficiency on listening comprehension items of the Test of English as a Foreign Language. Three two-way interactions were significant: proficiency by type of stimuli, type of stimuli by visual condition, and type of stimuli by time.…

Compares two language tests with different item types. The tests are the Dutch Reynell test and the BELL test. Both tests were administered to 64 Dutch kindergarten children with an average age of 70.3 months. Regression analyses indicate that item type does not contribute significantly to prediction of item difficulty, but the linguistic…

This article reports the results of using a three-layer back propagation artificial neural network to predict item difficulty in a reading comprehension test. Three classes of variables were examined: text structure, propositional analysis, and cognitive demand. Results demonstrate that the networks can consistently predict item difficulty. (JL)

This study examined the multiple true-false (MTF) test format in second language testing by comparing multiple-choice (MCQ) and multiple true-false (MTF) test formats in two language areas of general English: vocabulary and reading. Two counter-balanced experimental designs--one for each language area--were examined in terms of the number of MCQ…

One way of determining construct validity of vocabulary items in language tests is to interview subjects directly after taking the items to ascertain what is known about the target words in question. This approach was combined within the framework of lexical competency in a study of the behavior of lexical items on the Test of English as a Foreign…

Presents a study conducted to provide a comparative analysis of five item analysis indices using both IRT and non-IRT indices to describe the characteristics of flagged items and to investigate the appropriateness of logistic regression as an item analysis technique for further studies. The performance of five item analysis indices was examined.…