Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

In large-scale assessments, such as the National Assessment of Educational Progress (NAEP), plausible values based on Multiple Imputations (MI) have been used to estimate population characteristics for latent constructs under complex sample designs. Mislevy (1991) derived a closed-form analytic solution for a fixed-effect model in creating…

Test equating is a statistical procedure for adjusting for test form differences in difficulty in a standardized assessment. Equating results are supposed to hold for a specified target population (Kolen & Brennan, 2004; von Davier, Holland, & Thayer, 2004) and to be (relatively) independent of the subpopulations from the target population (see…

This study aimed at investigating grade four students learning at Jimma town, Ethiopia. A cross sectional survey method was employed. Students, teachers and principals were participants of the study. Standardized tests, questionnaire, interview and observation were the instruments used to collect authentic information from the research…

The National Assessment of Educational Progress; Second Science Assessment No-Show Study assessed the magnitude and causation of nonresponse biases. A No-Show is defined as an individual who was selected as a sample respondent but failed to be present for regular assessment of the 17-year-old group. The procedure whereby a sample of eligible…

An expanded multiple matrix sampling procedure to collect assessment data for health education courses in upper elementary and junior high schools is outlined. This supplement to the Cooperative Health Education Test provides data at the school building and district levels. (JMF)

The National Assessment of Educational Progress (NAEP) has completed two assessments of mathematics, the first conducted in 1972-73 and the second during 1977-78. Each assessment surveyed the mathematics achievement of American 9-, 13-, and 17-year-olds, using a deeply stratified, multi-stage probability sample design. This report documents…

Some sample data used in economic education research may be biased because missing observations may not be random. The authors explain a technique for estimating the relationship between students who fail to take standardized tests (the missing observations) and a core of general variables that might explain this behavior. (Author)

When evaluating the effectiveness of an educational program, multiple matrix sampling is particularly effective and efficient when the goal of the evaluation is estimating group (as opposed to individual) performance. The technique is described in some detail, with its advantages and disadvantages, and examples of its application are given.…

In 1982, the Educational Testing Service (ETS) proposed to implement a new, complex design for the National Assessment of Educational Progress (NAEP). The major features of this design are described in "A New Design for a New Era" (Messick, Beaton, and Lord, 1983). The purpose of this document is to describe the actual implementation of…

An alternative method for analyzing and evaluating prototype and revised instructional material is compared to traditional methods. The procedure and three approaches of sequential analysis are presented and advantages are discussed. References are included. (BK)

The National Assessment of Educational Progress (NAEP), like most large national surveys, employs a complex stratified multistage unequal probability sample. The design provides a rigorous justification for extending survey results to the entire U.S. target population. Developments in the analysis of data from complex surveys which provide a…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…

A stratified two-stage cluster sample design was chosen for the Texas Assessment Project and is described in this document. The sample was designed so as to achieve a chosen level of precision in item analysis data within each of the 20 geographic regions chosen for analysis. Four sampling frames were developed: 9-year-olds; sixth graders;…

Inferences that might be made from international educational assessments are discussed, considering issues of popular definition and sampling, assessment exercises, background variables, and statistical analyses. International assessments do tell something about academic achievement, but other approaches may provide more cost-effective information…