The main purpose of this study is to improve estimation efficiency in obtaining maximum marginal likelihood estimates of contextual effects in the framework of nonlinear multilevel latent variable model by adopting the Metropolis-Hastings Robbins-Monro algorithm (MH-RM). Results indicate that the MH-RM algorithm can produce estimates and standard…

The main purpose of this study is to improve estimation efficiency in obtaining maximum marginal likelihood estimates of contextual effects in the framework of nonlinear multilevel latent variable model by adopting the Metropolis-Hastings Robbins-Monro algorithm (MH-RM; Cai, 2008, 2010a, 2010b). Results indicate that the MH-RM algorithm can…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

This report examines the differences in impact estimates and standard errors that arise when these are derived using state achievement tests only (as pre-tests and post-tests), study-administered tests only, or some combination of state- and study-administered tests. State tests may yield different evaluation results relative to a test that is…

There has been little data collected on how much testing actually goes on in America's schools and how the results are used. So in the Spring of 2014, the Council staff developed and launched a survey of assessment practices. This report presents the findings from that survey and subsequent Council analysis and review of the data. It also offers…

Large sample standard errors for the Tucker method of linear equating under the common item nonrandom groups design are derived under normality assumptions as well as under less restrictive assumptions. Standard errors of Tucker equating are estimated using the bootstrap method described by Efron. The results from different methods are compared…

Sampling activities for Year 11 of the National Assessment of Educational Progress began in 1977 when plans were begun to Years 11-14. In March 1979 the sample was selected and allocated. In-school secondary sample selection activities were carried out during May through August, 1979, and in-school package assignment and field support activities…

This final report summarizes Year 11 quality check activities for the National Assessment of Educational Progress (NAEP). A probability sample of 40 schools was selected for quality check purposes from all three age classes. One regular school was selected for each District Supervisor at each Age Class. Quality check activities were conducted in…

This study addressed statistical techniques that might ameliorate some of the sampling problems currently facing states with small populations participating in State National Assessment of Educational Progress (NAEP) assessments. The study explored how the application of finite population correction factors to the between-school component of…

The National Assessment of Educational Progress in-school sampling design is a three-stage stratified design. Stratification variables include region, size of community and socioeconomic status. The three levels of sample selection are Primary Sampling Units (PSUs), schools and students. In general, two and sometimes three PSUs are selected from…

Beginning with the planning stages of the National Assessment of Educational Progress (NAEP), careful attention has been given to the design of efficient probability sampling methods for the selection of class-age respondents and the assignment of test packages. With these methods, it is possible for NAEP researchers to make relatively precise…

The correlations of each of the 22 tests in the Johnson O'Connor Research Foundation battery with all other tests in the battery are listed. Four fairly large samples are used, each including cases of one sex and a narrow age range. These cases come from a file of 3,555 examinees tested between June 1981 and the fall of 1982. The purpose of…

The National Assessment of Educational Progress (NAEP) measures the educational achievement of nationally representative samples of students in grades 4, 8, and 12. Local educational agencies tend to view the NAEP as a benchmark to which the educational achievement of their students can be compared. In particular, state departments of education…

This study provides empirical evidence about the sampling variability and generalizability (reliability) of a statewide performance assessment for grade six. Results for 600 students at individual and school levels indicate that task-sampling variability was the major source of measurement error. Rater-sampling variability was negligible. (SLD)

The National Education Association's Task Force on Testing has stated its opinion that standardized tests are overused. The task force suggests that the application of sampling techniques and a variety of alternatives to current testing practices would accomplish the same purposes. Representatives of the testing industry have indicated that the…