Evaluated a computer-delivered response type for measuring quantitative skill, the "Generating Examples" (GE) response type, which presents under-determined problems that can have many right answers. Results from 257 graduate students and applicants indicate that GE scores are reasonably reliable, but only moderately related to Graduate…

We make five over-time comparisons of student knowledge of civics and government: a) knowledge among 4th, 8th, and 11/12th graders between 1975/6 and 1998 using two separate NAEP trend assessments; b) knowledge over the same period by comparing responses to individual items asked in the 1975/6 and 1981/2 assessments to responses on identical…

This paper describes and evaluates a new type of multiple-choice test question that is relatively easy to construct and that challenges students' understandings of biological concepts. The questions involve a small narrative of scientific text that students must evaluate for accuracy. These are termed "You are the Teacher" questions because the…

The purpose of this study was to compare the effects of four item selection rules--(1) Fisher information (F), (2) Fisher information with a posterior distribution (FP), (3) Kullback-Leibler information with a posterior distribution (KP), and (4) completely randomized item selection (RN)--with respect to the precision of trait estimation and the…

Ability estimates yielded by the one- (1PL), two- (2PL), and three-parameter (3PL) models and the nominal response model (NRM) were compared with the number-right (NR) scoring model using items not susceptible to test-wiseness (NTW) and items susceptible to the ID1 test-wiseness strategy. These items were contained in grade 12 diploma examinations…

The problem of dimensionality with respect to Raven's Advanced Progressive Matrices (APM) specifically and, more generally, "g" or fluid intelligence, has been a long-standing issue. The present article reports two studies examining the dimensionality of both the original Set II of the APM (n = 506) and a short form (n = 644), using principal…

This research examined the degree of comparability and sources of incomparability of English and French versions of reading, mathematics, and science tests that were administered as part of a survey of achievement in Canada. The results point to substantial psychometric differences between the 2 language versions. Approximately 18% to 36% of the…

This study examined the relationship between the need for cognition, defined as the tendency to engage in effortful cognitive activity, and metacognition which is one's thinking about thinking and how these variables relate to intellectual task performance. Participants completed measures of need for cognition, metacognition, and problem-solved…

A popular design in large-scale educational assessments as well as any other type of survey is the balanced incomplete block design. The design is based on an item pool split into a set of blocks of items that are assigned to sets of "assessment booklets." This article shows how the problem of calculating an optimal balanced incomplete block…

Eight independent variables (differential item functioning [DIF] detection method, purification procedure, item response model, mean latent trait difference between groups, test length, DIF pattern, magnitude of DIF, and percentage of DIF items) were manipulated, and two dependent variables (Type I error and power) were assessed through…

Computer-assisted assessment (CAA) has become increasingly important in education in recent years. A variety of computer software systems have been developed to help assess the performance of students at various levels. However, such systems are primarily designed to provide objective assessment of students and analysis of test items, and focus…

[Gottfredson, L. S. (2003). Dissecting practical intelligence theory: Its claims and evidence. Intelligence, 31, 343-397.] provided a detailed critique of Sternberg's [Sternberg, R. J., Fotsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., Snook, S. A., Grigorenko, E. L. (2000). Practical intelligence in everyday life. New…

The Revised NEO Personality Inventory (NEO PI-R) was developed to operationalize the Five-Factor Model of Personality. Using correlational analysis and confirmatory and exploratory factor analysis, the present study investigates the facet structure of the domain of Agreeableness of the NEO-PI-R at the facet and item level to assess which is a more…

Incidental focus on form overtly draws learners' attention to linguistic items as they arise spontaneously--without prior planning--in meaning-focused interaction. This study examined the effectiveness of incidental focus on form in promoting second language (L2) learning. Seventeen hours of naturally occurring, meaning-focused L2 lessons were…

Introduction: Claude Steele's stereotype threat hypothesis proposed that negative group stereotypes increase individual anxiety levels, hurting performance. However, the role of anxiety in stereotype threat has not been fully explored. This study examined the hypothesis that experimental manipulation of stereotype threat would influence real-time…