Within an evidentiary framework for operationally defining academic English language proficiency (AELP), linguistic analyses of standards, classroom discourse, and textbooks have led to specifications for assessment of AELP. The test development process described here is novel due to the emphasis on using linguistic profiles to inform the …

This study used principles underlying item generation theory to posit competing perspectives about which features of situational judgment tests might enhance or impede consistent measurement across repeat test administrations. This led to 3 alternate-form development approaches (random assignment, incident isomorphism, and item isomorphism). The…

For decades traditional methods of testing have been criticized for saying relatively little reliably about students' ability as well as causing anxiety, which can negatively affect students' recall of learned information. The reform movement with its innovative approaches focusing on learner-centered education perceives assessment as an…

Computer adaptive testing is the study of scoring tests and questions based on assumptions concerning the mathematical relationship between examinees' ability and the examinees' responses. Adaptive student tests, which are based on item response theory (IRT), have many advantages over conventional tests. We use the least square method, a…

The Test of English as a Foreign Language (TOEFL) program is exploring a change in Section 3 of the TOEFL test that would replace the vocabulary subpart with additional reading comprehension questions. This study investigated the proposed revision in terms of the length and timing that would be necessary to address concerns of test speededness of…

The use of the Mantel-Haenszel statistic was investigated as a methodology for identifying differentially functioning items on the NTE Programs Core Battery. Retrospective analyses of the data collected over a 3-year period are reported for Black/White, Hispanic/White, and female/male comparisons in 50 samples ranging from 88 to 23,773 teacher…

When an examination consists, in whole or in part, of constructed response items, it is a common practice to allow the examinee to choose among a variety of questions. This procedure is usually adopted so that the limited number of items that can be completed in the allotted time does not unfairly affect the examinee. This results in the de facto…

The purpose of this study was to identify differentially functioning items on operational administrations of the Graduate Management Admission Test (GMAT) through the use of the Mantel-Haenszel statistic. Retrospective analyses of data collected over 3 years are reported for black/white and female/male comparisons for the Verbal and Quantitative…

This report describes how an advisory committee of health occupations education instructors from Missouri schools where tests were given statistically analyzed test items for health services assistant for reliability and validity and then revised the test items. Seventy-five sets of test item booklets were printed. Four test booklets and a…

The function of Mantel-Haenszel (MH) and logistic regression (LR) statistics with real data in detecting gender-based differentially functioning items (DIF) was investigated when sample size and criterion variable varied. The data base consisted of the item responses of a population of 183,356 Caucasians to the Math test of the ACT Assessment…

Examinee response times from a computerized adaptive test taken by 204 examinees taking a certification examination were analyzed using a hierarchical linear model. Two equations were posed: a within-person model and a between-person model. Variance within persons was eight times greater than variance between persons. Several variables…

The presence of multiple readings of a student response to a constructed-response item in a large-scale assessment requires a procedure for combining the ratings to obtain an item score. An alternative to the averaged item ratings that are usually used is the summing of ratings for each item. This study evaluated the effect of summing as opposed…

This digest describes some basics of the construction of multiple-choice tests. As a rule, the test maker should strive for test item stems (introductory questions or incomplete statements at the beginning of each item that are followed by the options) that are clear and parsimonious, answers that are unequivocal and chosen by the students who do…

Tests in social studies and integrated science given in Saint Vincent, Saint Lucia, Grenada, and Dominica were analyzed by the Organization for Co-operation in Overseas Development (OCOD) Comprehensive Teacher Training Program (CTTP) for discrimination, difficulty, and reliability, as well as other characteristics. There were 767 examinees for the…

There is a considerable interest at Educational Testing Service (ETS) to include performance-based, natural language constructed-response items on standardized tests. Such items can be developed, but the projected time and costs required to have these items scored by human graders would be prohibitive. In order for ETS to include these types of…