Long-running surveys need a systematic way to reflect social change and to keep items relevant to respondents, especially when they ask about controversial subjects, or they threaten the items' validity. We propose a protocol for updating measures that preserves content and construct validity. First, substantive experts articulate the current and…

Many Flynn effect (FE) studies compare scores across different editions of Wechsler's IQ tests. When construct changes are introduced by the test developers in the new edition, however, the presumed generational effects are difficult to untangle from changes due to test content. To remove this confound, we use the same edition of Wechsler…

The Test Of Astronomy STandards (TOAST) is a comprehensive assessment instrument designed to measure students' general astronomy content knowledge. Built upon the research embedded within a generation of astronomy assessments designed to measure single concepts, the TOAST is appropriate to measure across an entire astronomy course. The TOAST's…

In the fall of 2014, the University at Buffalo created a new universal Student Evaluation of Teaching (SET). The purpose of the present study was to establish the construct validity of SET items. Rasch analyses of data from 7 semesters (N=203,194 students) revealed problems with item fit indices and threshold distances. Changes to items and…

This study aimed to develop an instrument, named the inquiry-based laboratory classroom environment instrument (ILEI), for assessing senior high-school science students' preferred and perceived laboratory environment. A total of 262 second-year students, from a senior-high school in Taiwan, were recruited for this study. Four stages were included…

The aim of this mixed methods study is to identify the underlying structure of the construct of Foreign Language Anxiety in integrated listening-to-speak tasks. First, the analysis of the qualitative interviews with six postsecondary ESL learners reveals that anxiety for integrated speaking stems from four different factors: "listening,"…

This chapter introduces theory that undergirds the role of testing adaptations in assessment, provides examples of item modifications and testing accommodations, reviews research relevant to each, and introduces a new paradigm that incorporates opportunity to learn (OTL), academic enablers, testing adaptations, and inferences that can be made from…

Although classroom assessment is one of the most frequent practices carried out by teachers in all educational programs, limited research has been conducted to investigate the dependability and validity of criterion-referenced tests (CRTs). The main purpose of this study is to develop a criterion-referenced test for first-year Japanese university…

The purpose of this study was to develop a Korean College Stress Inventory (KCSI), which is designed to measure Korean college students' experiences and symptoms of career stress. Even though there have been numerous scales related to career issues, few scales measure the career stress construct and its dimensions. Factor structure, internal…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

Lissitz and Samuelsen (2007) propose a new framework for validity theory and terminology, emphasizing a shift in theory and practice toward issues of test content rather than constructs. The author of this article argues that several of Lissitz and Samuelsen's critiques of validity theory focus on previously considered, but subsequently discarded,…

This study was designed to address two frequent criticisms of the SAT essay--that essay length is the best predictor of scores, and that there is an advantage in using more "sophisticated" examples as opposed to personal experience. The study was based on 2,820 essays from the first three administrations of the new SAT. Each essay was…

The purpose of this paper is to clarify the seemingly discrepant views of test theorists and test developers about terminology related to the evaluation of test content. The origin and evolution of the concept of content validity are traced, and the concept is reformulated in a way that emphasizes the notion that content domain definition,…

An expanded version of the method of content evaluation proposed by S. G. Sireci and K. F. Giesinger (1992) was evaluated with respect to a national licensure examination and a nationally standardized social studies achievement test. Two groups of 15 subject-matter experts rated the similarity and content relevance of the items. (SLD)

Fifty education majors in two sections responded to an Attitudes toward Mathematics and Its Teaching (ATMAT) scale. Results with two psychometric models, classical true-score theory and the one-parameter Rasch model, supported the ATMAT's reliability, content and construct validity, and invariance over three time points. (SLD)