The purpose of this study was to demonstrate the effectiveness level of the Provus-Alkin-amalgamation evaluation application based on Weighted-Product-"Rwa-Bhineda" modification through the results of field trials. This evaluation application was an evaluation tool formed by a combination of educational evaluation models (Provus model…

The English Learner population (EL) in the United States continues to grow, with more than 5 million students currently identified as English Learner students. While districts and schools were held accountable for their English Learner students under the No Child Left Behind Act[superscript 4], the Every Student Succeeds Act (ESSA)[superscript 5]…

Propensity score matching and weighting methods are often used in observational effectiveness studies to reduce imbalance between treated and untreated groups on a set of potential confounders. However, much of the prior methodological literature on matching and weighting has yet to examine performance for scenarios with a majority of treated…

Reweighting a distribution to minimize a distance to a target distribution is a powerful and flexible strategy for estimating a wide range of causal effects, but can be challenging in practice because optimal weights typically depend on knowledge of the underlying data generating process. In this paper, we focus on design-based weights, which do…

Background: A growing number of surveys are conducted online where respondents can choose to complete the questionnaire (Lehdonvirta et al., 2020). As respondents are self-selected, there is potential that the respondents will not be an accurate representation of the population. For example, white people are disproportionately more likely to…

The simple average of student growth scores is often used in accountability systems, but it can be problematic for decision making. When computed using a small/moderate number of students, it can be sensitive to the sample, resulting in inaccurate representations of growth of the students, low year-to-year stability, and inequities for…

Structural equation modeling (SEM) is a widely used technique for studies involving latent constructs. While covariance-based SEM (CB-SEM) permits estimating the regression relationship among latent constructs, the parameters governing this relationship do not apply to that among the scored values of the constructs, which are needed for…

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and…

Purpose: In grading, one of the most common errors is made in combining two or more different test scores. This study aimed to investigate the agreement of grades calculated by weighting raw scores and standard scores. Research Methods: In this simulation study, data were simulated for midterm and final measurements. Nine conditions [3 (class…

The traditional points system and subsequent Grade Point Average (GPA) in education perpetuates an evaluation of academic performance which reflects arbitrary weighting of assignments and/or assessments. As such, GPAs which are calculated using a traditional points system are not unidimensional in their design. The baseball batting and slugging…

The purpose of this occasional paper is to compare point estimates and trends included in the Monitoring the Future (MTF) Panel Study annual report: "National Data on Substance Use among Adults Ages 19 to 60, 1976-2021" prepared using historical post-stratification weights, with the same point estimates and trends obtained when using MTF…

Statistical process control (SPC) charts have been widely used in the field of educational measurement. The cumulative sum (CUSUM) is an established SPC method to detect aberrant responses for educational assessments. There are many studies that investigated the performance of CUSUM in different test settings. This paper describes the CUSUM…

This three-article dissertation aims to address three methodological challenges to ensure comparability in educational research, including scale linking, test equating, and propensity score (PS) weighting. The first study intends to improve test scale comparability by evaluating the effect of six missing data handling approaches, including…

This study evaluated the impact of subgroup weighting for equating through a common-item anchor. We used data from a single test form to create two research forms for which the equating relationship was known. The results showed that equating was most accurate when the new form and reference form samples were weighted to be similar to the target…

Ramsay and Wiberg used a new version of item response theory that represents test performance over nonnegative closed intervals such as [0, 100] or [0, n] and demonstrated that optimal scoring of binary test data yielded substantial improvements in point-wise root-mean-squared error and bias over number right or sum scoring. We extend these…