In this experimental study, 20 multiple-choice test items from the Massachusetts Grade 5 science test were linguistically simplified, and original and simplified test items were administered to 310 English learners (ELs) and 1,580 non-ELs in four Massachusetts school districts. This study tested the hypothesis that specific linguistic features of…

Throughout the world, tests are administered to some examinees who are not fully proficient in the language in which they are being tested. It has long been acknowledged that proficiency in the language in which a test is administered often affects examinees' performance on a test. Depending on the context and intended uses for a particular…

The International Test Commission's "Guidelines for Translating and Adapting Tests" (2010) provide important guidance on developing and evaluating tests for use across languages. These guidelines are widely applauded, but the degree to which they are followed in practice is unknown. The objective of this study was to perform a…

Over the past four decades, there has been incremental growth in computer-based testing (CBT) as a viable alternative to paper-and-pencil testing. However, the transition to CBT is neither easy nor inexpensive. As Drasgow, Luecht, and Bennett (2006) noted, many design engineering, test development, operations/logistics, and psychometric changes…

Seven specific guidelines included in the taxonomy proposed by T. Haladyna and S. Downing (1998) for writing multiple-choice test items were evaluated. These specific guidelines are: (1) avoid the complex multiple-choice, K-type format; (2) state the stem in question format; (3) word the stem positively; (4) avoid the phrase "all of the…

Free-response (FR) item formats, such as essay questions, are popular in educational assessment. The criticisms against FR items are that they are more expensive to score, take up more testing time, provide less content coverage, and are less reliable than multiple-choice (MC) items. For these reasons, FR items are often combined with MC items.…

The purpose of this paper is to clarify the seemingly discrepant views of test theorists and test developers about terminology related to the evaluation of test content. The origin and evolution of the concept of content validity are traced, and the concept is reformulated in a way that emphasizes the notion that content domain definition,…

Test developers continue to struggle with the technical and logistical problems inherent in assessing achievement across different languages. Many testing programs offer separate language versions of a test to evaluate the achievement of examinees in different language groups. However, comparisons of individuals who took different language…

Reviews and illustrates some of the current technological developments in computer-based testing, focusing on novel item formats and automated scoring methodologies. The review shows a number of innovations being researched and implemented. (SLD)

A new method for evaluating the content representation of a test is illustrated. Item similarity ratings were obtained from three content domain experts to assess whether ratings corresponded to item groupings specified in the test blueprint. Multidimensional scaling and cluster analysis provided substantial information about the test's content…

Studied a method for investigating the equivalence of translated-adapted items using bilingual test takers through item response theory. Results from an English-Turkish course evaluation form completed by 688 Turkish students indicate that the methodology is effective in flagging items that function differentially across languages and informing…

International comparative educational studies make use of test instruments originally developed in English by international panels of experts, but that are ultimately administered in the language of instruction of the students. The comparability of the different language versions of these assessments is a critical issue in validating the…

Many students in the United States who are required to take educational tests are not fully proficient in English. To address this problem, a state-mandated testing program created dual language English-Spanish versions of some of their tests. In this study, the psychometric properties of the English and dual language versions of a fourth-grade…

Educational tests used for accountability purposes must represent the content domains they purport to measure. When such tests are used to monitor progress over time, the consistency of the test content across years is important for ensuring that observed changes in test scores are due to student achievement rather than to changes in what the test…

An expanded version of the method of content evaluation proposed by S. G. Sireci and K. F. Giesinger (1992) was evaluated with respect to a national licensure examination and a nationally standardized social studies achievement test. Two groups of 15 subject-matter experts rated the similarity and content relevance of the items. (SLD)