The increasing volume of large-scale assessment data poses a challenge for testing organizations to manage data and conduct psychometric analysis efficiently. Traditional psychometric software presents barriers, such as a lack of functionality for managing data and conducting various standard psychometric analyses efficiently. These challenges…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

This case study provides an account of my experiences and lessons learned in designing and implementing a survey research. The research was part of my doctoral thesis in educational measurement and quantitative methods. The purpose of the study was to examine the effectiveness of a Completely Structured Training compared with Partially Structured…

Purpose: The purpose of this paper is to examine relationship between teacher competences and organizational citizenship behavior (OCB) with empowerment as a mediating factor. Design/methodology/approach: The study took a cross-sectional descriptive and analytical design. Using cluster and random sampling procedures, data were obtained from 383…

In this paper, we examine the validity and precision of two nonexperimental study designs (NXDs) that can be used in educational evaluation: the comparative interrupted time series (CITS) design and the difference-in-difference (DD) design. In a CITS design, program impacts are evaluated by looking at whether the treatment group deviates from its…

This dissertation reports the results of three studies and a pilot study. The first study was a Monte Carlo validation study that examined the accuracy of a new visual inspection method, the semi-interquartile range method. Results of the study indicated that this method had lower levels of power than a previously validated method, the…

The Louisiana Scholarship Program (LSP) is a statewide initiative offering publicly-funded vouchers to enroll in local private schools to students in low-performing schools with family income no greater than 250 percent of the poverty line. Initially established in 2008 as a pilot program in New Orleans, the LSP was expanded statewide in 2012.…

Purpose: The purpose of this study was to examine the effectiveness of the Multi-Subject Learning Survey (MSLS) as an assessment instrument for graduate students. Methodology: The MSLS was administered as a pre-test to 124 students enrolled in the introductory course for the Master of Science in Library Science degree at an American Library…

Purpose: District effectiveness research (DER) is an emerging field concerned with identifying the organizational structures, administration, and leadership practices at the school district level that help districts find success with all of their students across the schools within the system. This work has mirrored much of the early school…

An explanation of the misuse of statistical significance testing and the true meaning of "significance" is offered. Literature about the criticism of current practices of researchers and publications is reviewed in the context of tests of significance. The problem under consideration occurs when researchers attempt to do more than just establish…

This paper discusses how an experimental design can be applied to a large-scale evaluation. The purpose of the study described is to assess the success of the National Science Foundation (NSF) in achieving its goal for five comprehensive projects. This paper is divided into four main sections: (1) design; (2) sampling procedures; (3) data…

This paper discusses different perspectives associated with the use of multiple measures in educational assessment and explores some of the technical considerations for selecting and combining multiple measures. The paper concludes with an example of the use of multiple measures with regard to the recent No Child Left Behind (NCLB) Act of 2002.…

The National Assessment of Educational Progress (NAEP), like most large national surveys, employs a complex stratified multistage unequal probability sample. The design provides a rigorous justification for extending survey results to the entire U.S. target population. Developments in the analysis of data from complex surveys which provide a…

This paper examines how controlling statistically for baseline covariates (especially pretests) improves the precision of studies that randomize schools to measure the impacts of educational interventions on student achievement. Part I of the paper introduces the concepts, issues, and options involved. Parts II and III present empirical findings…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…