Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly…

The item wording (or keying) effect is respondents' differential response style to positively and negatively worded items. Despite decades of research, the nature of the effect is still unclear. This article proposes a potential reason; namely, that the item wording effect is scale-specific, and thus findings are applicable only to a particular…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedure item difficulty and dispersion parameters are…

When memory is tested, researchers are often interested in the items that were correctly recalled or recognized, while ignoring or factoring out trials where one "recalls" or "recognizes" a nonstudied item. However, intrusions and false alarms are more than nuisance data and can provide key insights into the memory system. The…

Background: A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals in order to better understand conditioning factors of test-taking disengagement: First, a model-based approach was used to…

Item development is a time- and resource-intensive process. Automatic item generation integrates cognitive modeling with computer technology to systematically generate test items. To date, however, items generated using cognitive modeling procedures have received limited use in operational testing situations. As a result, the psychometric…

The purpose of this study was to evaluate the adequacy of three cognitive models, one developed by content experts and two generated from student verbal reports for explaining examinee performance on a grade 3 diagnostic mathematics test. For this study, the items were developed to directly measure the attributes in the cognitive model. The…

Automatic item generation represents a relatively new but rapidly evolving research area where cognitive and psychometric theories are used to produce tests that include items generated using computer technology. Automatic item generation requires two steps. First, test development specialists create item models, which are comparable to templates…

New ways are proposed to look at data from the Canadian School Achievement Indicators Program (SAIP) using item response theory (IRT) modeling. The focus is on the traditional test of the SAIP 1997 mathematics study. The test is two-staged in that the first 15 items, of median difficulty, were to be completed beforehand by all students as a…

Thirty-six testwise and 41 test-naive high school seniors in British Columbia (Canada) were tested to determine their abilities to apply selected test wiseness principles according to a proposed model of test-taking behavior. To apply the testwiseness strategy, students first needed knowledge of the content tested and test item content. (SLD)

In preparation for a study of essay questions and other forms of open-ended exercises in the California Golden State Examination for biology, the functioning of open-ended biology items in another examination was explored. The Golden State Examination program offers honors credit to students who wish to qualify for admission to programs in…

This booklet provides samples of student writing that exemplify the standards that governed the scoring of student essays on the June 1988 Social Studies 30 Diploma Examination. Copies of the two essay assignments are provided, and five student essays are included for both topics, covering the full range of student writing. Commentaries…

The Second International Mathematics Study was conducted in 20 countries under the sponsorship of the International Association for Evaluation of Educational Achievement (IEA). Among the instruments used in this study was a questionnaire investigating student attitudes about school, instruction, and mathematics. The fit of an a priori model that…

Many significant changes in perspective have to take place before efforts to learn the content and capabilities of children's minds can hold much sway in educational testing. The language of testing, especially of high stakes testing, remains firmly in the realm of "behaviors", "performance" and "competency" defined…

This booklet provides samples of student writing that exemplify the standards that governed the scoring of student essays on the June 1988 English 30 Diploma Examination. Both major and minor writing assignments for seven students are provided, with examples of the full range of student writing. Commentaries accompanying the essays highlight…