The Partnership for Assessment of Readiness for College and Careers (PARCC) is a group of states working together to develop a set of assessments that measure whether students are on track to be successful in college and careers. Administrations of the PARCC assessment included three Prose Constructed Responses (PCR), one per task for English…

This research report presents a summary of research and development efforts devoted to creating scoring models for automatically scoring spoken item responses of a pilot administration of the Test of English-for-Teaching ("TEFT"™) within the "ELTeach"™ framework.The test consists of items for all four language modalities:…

Being driven by the hypothetico-deductive mode of inquiry, previous studies have tested the effectiveness of theory-driven interventions under controlled experimental conditions to come up with universally applicable generalizations. To make a case in the opposite direction, this data-driven study aims at uncovering techniques and strategies…

Stewart questioned Beglar's use of Rasch analysis of the Vocabulary Size Test (VST) and advocated the use of 3-parameter logistic item response theory (3PLIRT) on the basis that it models a non-zero lower asymptote for items, often called a "guessing" parameter. In support of this theory, Stewart presented fit statistics derived from…

Multiple-choice tests such as the Vocabulary Levels Test (VLT) are often viewed as a preferable estimator of vocabulary knowledge when compared to yes/no checklists, because self-reporting tests introduce the possibility of students overreporting or underreporting scores. However, multiple-choice tests have their own unique disadvantages. It has…

Cloze testing and language performance is discussed as are two techniques for awarding partial credit: the quick performance measurement and feedback technique and the three-stage scoring hierarchy for partial credit. A figure and tables are included. (EJS)

A recent multidimensional scaling analysis of the Test of English-as-a-Foreign-Language (TOEFL) item response data identified clusters of items in the test sections that, being more homogeneous than their parent sections, might be better for diagnostic use. The analysis was repeated using different scoring techniques. Results diverged only for…

This study assessed the invariance in the factor structure of the "Test of English as a Foreign Language"™ Internet-based test (TOEFL® iBT) across subgroups of test takers who differed in native language and exposure to the English language. The subgroups were defined by (a) Indo-European and Non-Indo-European language family, (b)…

The recent psycholinguistic-sociolinguistic trend in foreign language teaching indicates a shift of interest away from quantitative data towards qualitative information of different kinds. Validity and relevance are stressed and so new test formats are demanded, as well as new methods of dealing with student answers. Item analysis techniques used…

A project is described that was undertaken to investigate: (1) how long a cloze test has to be to achieve optimum reliability without wasting anyone's time; and (2) how cloze tests should be scored in order to obtain maximum reliability. The literature recommended 50 deletions in order to provide for an adequate sample of examinee's abilities; it…

Reviews the place of rating scales in second-language measurement and summarizes some of the problems associated with them. Standard and alternative scales were studied. High agreement among raters can be achieved even under conditions not favorable to high interrater reliability. The full range of score categories are effectively utilized. (17…

A study was undertaken to develop a partial credit scheme for scoring cloze-type questions on an English collocation test, obtain construct validity evidence for the test and the scoring scheme using the Rasch Partial Credit Model, and compare partial credit scoring with the more commonly used dichotomous scoring with the same test instrument.…

The Test of English as a Foreign Language (TOEFL) was used in this study, which attempted to develop a new methodology for assessing the speededness of right-scored tests. Traditional procedures of assessing speededness have assumed that the test is scored under formula-scoring instructions; this approach is not always appropriate. In this study,…