The present paper questions the content validity of the eight creativity-related self-report scales available in PISA 2022's context questionnaire and provides a set of considerations for researchers interested in using these indexes. Specifically, we point out some threats to the content validity of these scales (e.g., "creative thinking…

The Programme for International Assessment (PISA) is an important cross-national study of 15-year olds academic achievement. Although it has traditionally been conducted using paper-and-pencil tests, the vast majority of countries will use computer-based assessment from 2015. In this paper, we consider how cross-country comparisons of children's…

Variation in test performance among examinees from different regions or national jurisdictions is often partially attributed to differences in the degree of content correspondence between local school or training program curricula, and the test of interest. This posited relationship between test-curriculum correspondence, or "alignment,"…

A primary goal of standards-based statewide achievement tests is to classify students into achievement levels that enable valid inferences about student content area knowledge and skill. Explicating how knowledge and skills are expected to differ in complexity in achievement level descriptors, and how that complexity is related to empirical item…

The purpose of this document is to provide background information that will be useful in interpreting the 2007 results from the Trends in International Mathematics and Science Study (TIMSS) by comparing its design, features, framework, and items with those of the U.S. National Assessment of Educational Progress and another international assessment…

Wilcox has described three probability models which characterize a single test item in terms of a population of examinees (ED 156 718). This note indicates indicates that similar models can be derived which characterize a single examinee in terms of an item domain. A numerical illustration is given. (Author/JKS)

Traditionally, comparable content-area forms and levels of a multi-level, multi-form achievement test series have been equated using the equipercentile method. There is some evidence to indicate, however, that better procedures are needed. A linking procedure is being used to equate simultaneously 40 different test forms of an achievement test:…

Bilevel dimensionality of probability was examined via factor analysis, Rasch latent trait analysis, and classical item analysis. Results suggest that when nonstandardized measures are the criteria for achievement, relying solely on estimates of content validity may lead to erroneous interpretation of test score data. (JKS)

The "Instructional Improvement Digest" communicates advisory information about practical courses of action that can be implemented by teachers and administrators to improve key areas of school instruction. The series digest topics draws upon inquiry associated with the Southwest Regional Laboratory for Educational Research and Development's…

Procedures for conducting functional level testing have been available for use by practitioners for some time. However, the Title I Evaluation and Reporting System (TIERS), developed in response to the educational amendments of 1974 to the Elementary and Secondary Education Act (ESEA), has provided the impetus for widespread adoption of this…

Because it is difficult to ascertain the dimensionality of a test composed of binary items through the use of factor analysis alone, a method is proposed that combines item characteristic curve (ICC) theory with factor analysis. Factor structure of tetrachoric correlations is distorted by non-normal distribution of ability. Item characteristics…

A review of the research on item writing, item format, test instructions, and item readability indicated the importance of instrument structure in the interpretation of test data. The effect of failing to consider these areas on the content validity of achievement test scores is discussed. (Author/GK)

The sampling of test items found within the Science portion of selected standardized achievement test batteries is examined. The lack of sensitivity of such items to the curriculum changes often attempted in Earth Science is demonstrated. The following five standardized achievement test batteries commonly administered to pupils in grades 7-9 were…

Extended caution indices (ECI) specify the degree of confidence that can be placed in an individual's test score by analyzing patterns of item response. Among the most promising of such indices are the standardized ECIs. Contrary to the literature, several instances were found, in a previous study, of nonnormal distributions of ECIs with samples…

The Ohio Vocational Achievement Tests are specially designed instruments for use by teachers, supervisors, and administrators to evaluate and diagnose vocational achievement for improving instruction in secondary vocational programs at the 11th and 12th grade levels. This guide explains the Ohio Vocational Achievement Tests and how they are used.…