Describes a process for evaluating the effectiveness of formal courses or segments of training instruction, including lessons or modules. Topics discussed include focusing on objectives; objective and test item evaluation; on-the-job training; task levels; transfer of training; presentation evaluation; and evaluation of instructional…

Discusses multiple-choice test questions, their advantages, important features, well and poorly written items, and the need to assess students' ability to use higher levels of learning. Cites two ways to include higher level learning skills into the science curriculum and eight rules to consider when constructing multiple-choice tests. (RT)

Describes the use of Rasch item statistics for curricular analysis and instructional decision making. Uses a case study of application of the Rasch model in the pilot testing of items selected for criterion-referenced tests developed for grades one through eight in Charleston (South Carolina) schools. (SLD)

The validity of the assumptions under which the ordinal true score test theory was derived was examined using (1) simulation based on classical test theory; (2) a long empirical test with data from 321 sixth graders; and (3) an extensive simulation with 480 datasets based on the 3-parameter model. (SLD)

The difference between item bias and item impact and the way they relate to item validity are discussed from a multidimensional item response theory perspective. The Mantel-Haenszel procedure and the Simultaneous Item Bias strategy are used in a Monte Carlo study to illustrate detection of item bias. (SLD)

A procedure is presented for examining the influence of instruction on responses to test items by extending item response theory to incorporate variables illustrating different amounts of opportunity to learn. Data from the Second International Mathematics Study (grade 8 scores for about 7,000 students) illustrate the discussion. (SLD)

The use of restricted item response models to test hypotheses regarding item difficulty ordering and slope uniformity was demonstrated in a study in which 597 algebra students were asked to solve word problems reflecting various types of cognitive processing. Benefits and limitations of the procedures are discussed. (SLD)

Describes a method for assessing the quality of translations based on item response theory (IRT). Results from the IRT technique with French and Chinese versions of a scale measuring individualism-collectivism for samples of 250 U.S., 357 French, and 290 Chinese undergraduates show how several biased items are detected. (SLD)

The degree to which between-group differences in performance on individual questions on the NTE Professional Knowledge Test conform to expectations was studied for 19,307 examinees who had taken the NTE Core Battery. Some items are quite consistent with the intended test interpretation test, whereas others are less consistent. (SLD)

The usefulness of latent class modeling in addressing several measurement issues is demonstrated via a study of 74 good and 74 poor readers in grades 5 and 6. Procedures were particularly useful for assessing the hierarchical relation among skills and for exploring issues related to item domains. (SLD)

Assessments in English that are constructed for native English speakers may not provide valid inferences about the achievement of English language learners (ELLs). The linguistic complexity of the test items that are not related to the content of the assessment may increase the measurement error, thus reducing the reliability of the assessment.…

Bayesian networks are frequently used in educational assessments primarily for learning about students' knowledge and skills. There is a lack of works on assessing fit of Bayesian networks. This article employs the posterior predictive model checking method, a popular Bayesian model checking tool, to assess fit of simple Bayesian networks. A…

We contend that generalizability (G) theory allows the design of psychometric approaches to testing English-language learners (ELLs) that are consistent with current thinking in linguistics. We used G theory to estimate the amount of measurement error due to code (language or dialect). Fourth- and fifth-grade ELLs, native speakers of…

This study seeks to determine whether item features are related to observed differences in item difficulty (DIF) between computer- and paper-based test delivery media. Examinees responded to 60 quantitative items similar to those found on the GRE general test in either a computer-based or paper-based medium. Thirty-eight percent of the items were…

This paper reports replications of studies of implicit artificial grammar (AG) learning and explicit series-solution learning with experienced second language learners in order to examine their population and content generalizability. As found by Reber, Walkenfeld, and Hernstadt (1991), there was significantly greater variance in explicit compared…