In alignment studies, expert raters evaluate assessment items against standards and ratings are used to compute various alignment indices. Questions about rater reliability, however, are often ignored or inadequately addressed. This paper reports the results of a generalizability theory study of cognitive demand and rigor ratings of assessment…

College students participated in a study on the "psychology of note taking" during which they took notes on video content and later completed a multiple-choice test on the material. Researchers assigned 71 participants to either the ringing condition (the video was disrupted by a ringing cell phone) or the control condition (no cell phone rings…

The traditional way of equating the scores on a new test form X to those on an old form Y is equipercentile equating for a population of examinees. Because the population is likely to change between the two administrations, a popular approach is to equate for a "synthetic population." The authors of the articles in this issue of the…

Background: From the perspective of self-regulated learning, the interplay between learners' individual characteristics and the context of testing have been emphasized for assessing learning outcomes. Aims: The present study examined metacognitive processes in children's test-taking behaviour and explored their impacts on performance. Further, it…

A questionnaire used in student evaluations of interdisciplinary courses during six semesters contained two Likert items stated in a direct negative mode which were embedded in a questionnaire (14-18 items) in which the remaining items were phrased in a direct positive mode. In the seventh semester and thereafter, the two negative items were…

Gender differences in aggression have typically been based on studies utilizing a mean difference method. From a measurement perspective, this method is inherently problematic unless an aggression measure possesses comparable validity across gender. Stated differently, establishing measurement invariance on the measure of aggression is…

In most operational computerized adaptive testing (CAT) programs, the Sympson-Hetter (SH) method is used to control the exposure of the items. Several modifications and improvements of the original method have been proposed. The Stocking and Lewis (1998) version of the method uses a multinomial experiment to select items. For severely constrained…

An item model serves as an explicit representation of the variables in an assessment task. An item model includes the "stem", "options", and "auxiliary information". The "stem" is the part of an item which formulates context, content, and/or the question the examinee is required to answer. The "options" contain the alternative answers with one…

In his analysis of the pseudoword effect, [Greene, R.L. (2004). Recognition memory for pseudowords. "Journal of Memory and Language," 50, 259-267.] suggests nonwords can feel more familiar that words in a recognition context if the orthographic features of the nonword match well with the features of the items presented at study. One possible…

Students who are deaf or hard of hearing (SDHH) often need accommodations to participate in large-scale standardized assessments. One way to bridge the gap between the language of the test (English) and a student's linguistic background (often including American Sign Language [ASL]) is to present test items in ASL. The specific aim of this project…

Most professors use examinations as an important assessment tool to aid in determining the level of student subject matter comprehension. We also use the feedback from examinations as an indicator of the appropriateness and effectiveness of the teaching methodologies we are utilizing in the classroom. This paper is a follow-up to a 2006-2007 study…

This article presents procedures and results from a 2-year project developing science key vocabulary (KV) short tests suitable for progress monitoring Grade 5 science in Texas public schools using computer-generated, -administered, and -scored assessments. KV items included KV definitions and important usages in a multiple-choice cloze format. A…

This survey explores the post-graduation outcomes of university students with disabilities. It gathers data on their employment, independent living, community participation/social integration, and supports received by adult disability agencies. It also captures their perceptions about their quality of life. (Contains 1 figure.) [This survey tool…

Multilevel data often exist in educational studies. The focus of this study is to consider differential item functioning (DIF) for dichotomous items from a multilevel perspective. One of the most often used methods for detecting DIF in dichotomously scored items is the Mantel-Haenszel log odds-ratio. However, the Mantel-Haenszel reduces the…

The structure and magnitude of sex differences in interpersonal problems across several data sets were examined, guided by the interpersonal circumplex model and the structural summary method. Data were self-reported interpersonal difficulties, assessed with the 64-item version of the Inventory of Interpersonal Problems (IIP; L. M. Horowitz, S. E.…