Background/Context: Observational studies often employ regression discontinuity (RD) designs and multiple control-group designs to explore the causal quantities of interest. RD designs assess policy and program effectiveness by assigning subjects to treatment based on whether they exceed a pre-defined cutoff. RD designs are classified into two…

Background/Context: Some observational studies involve multiple layers of treatment selection, specifically in the context of the extended time accommodation (ETA) for English language learners (ELLs). In ETA settings, the first selection occurs due to the eligibility rule, where students whose ELL English proficiency is below a certain threshold…

Regression discontinuity (RD) designs are commonly used for program evaluation with continuous treatment assignment variables. But in practice, treatment assignment is frequently based on ordinal variables. In this study, we propose an RD design with an ordinal running variable to assess the effects of extended time accommodations (ETA) for…

The variance formula derived for a two-stage sampling design without replacement employs the joint inclusion probabilities in the first-stage selection of clusters. One of the difficulties encountered in data analysis is the lack of information about such joint inclusion probabilities. One way to solve this issue is by applying Hájek's…

Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous…

The 2011 NAEP-TIMSS linking study conducted by the National Center for Education Statistics (NCES) was designed to predict Trends in International Mathematics and Science Study (TIMSS) scores for the U.S. states that participated in 2011 National Assessment of Educational Progress (NAEP) mathematics and science assessment of eighth-grade students.…

For more than four decades the National Assessment of Educational Progress (NAEP) has tracked the achievement of U.S. students in major academic subjects. This national resource is the only assessment that states and now many urban districts can look to as an objective yardstick of their performance over time, relative to national benchmarks, and…

Since its inception, the National Center for Education Statistics (NCES) has been committed to the practice of documenting its statistical methods for its customers and of seeking to avoid misinterpretation of its published data. The reason for this policy is to assure customers that proper statistical standards and techniques have been observed,…

The National Assessment Governing Board adopted a Program of Preparedness Research in March 2009. Several categories of research studies were recommended to produce results for reporting 12th grade preparedness for the 2009 grade 12 National Assessment of Educational Progress (NAEP) in reading and mathematics. The categories included content…

The trends and findings from a descriptive analysis of 25 years of research studies examining the effectiveness of technology to support the compositional writing of students with learning and academic disabilities are presented. A corpus of 85 applied research studies of writing technology effectiveness was identified from among 249 items in the…

A popular design in large-scale educational assessments is the balanced incomplete block design. The design assumes that the item pool is split into a set of blocks of items that are assigned to assessment booklets. This paper shows how the technique of 0-1 linear programming can be used to calculate a balanced incomplete block design. Several…

Explains how schools and students are selected for the National Assessment of Educational Progress (NAEP) and why high participation rates are important. Describes the multistage sampling method, which allows the National Center for Education Statistics to give an accurate picture of student performance while keeping the burden on schools and…

The 1994 National Assessment of Educational Progress (NAEP) monitored the performance of students in American schools in reading, U.S. history, geography, mathematics, science, and writing. The sample involved more than 135,000 public and private school students who were 9, 13, or 17 years old and/or in grades 4, 8, and 12. This technical report…

Describes and assesses missing data methods currently used to analyze data from matrix sampling designs implemented by the National Assessment of Educational Progress. Several improved methods are developed, and these models are evaluated using an EM algorithm to obtain maximum likelihood estimates followed by multiple imputation of complete data…

The National Assessment of Educational Progress (NAEP) has been the subject of many evaluations during its history. Those evaluations are reviewed, focusing on external evaluations, and the influences of recommendations made in the evaluations are discussed. The report concludes that many of the recommendations of evaluators have been heeded by…