The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit with panel data. Two challenges arise with higher frequency data (e.g., monthly versus yearly): (1) achieving excellent pre-treatment fit is typically more challenging; and (2) overfitting to noise is more likely. Aggregating data…

The simple average of student growth scores is often used in accountability systems, but it can be problematic for decision making. When computed using a small/moderate number of students, it can be sensitive to the sample, resulting in inaccurate representations of growth of the students, low year-to-year stability, and inequities for…

Structural equation modeling (SEM) is a widely used technique for studies involving latent constructs. While covariance-based SEM (CB-SEM) permits estimating the regression relationship among latent constructs, the parameters governing this relationship do not apply to that among the scored values of the constructs, which are needed for…

The traditional points system and subsequent Grade Point Average (GPA) in education perpetuates an evaluation of academic performance which reflects arbitrary weighting of assignments and/or assessments. As such, GPAs which are calculated using a traditional points system are not unidimensional in their design. The baseball batting and slugging…

Ramsay and Wiberg used a new version of item response theory that represents test performance over nonnegative closed intervals such as [0, 100] or [0, n] and demonstrated that optimal scoring of binary test data yielded substantial improvements in point-wise root-mean-squared error and bias over number right or sum scoring. We extend these…

Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student…

Structural equation modeling (SEM) and path analysis using composite-scores are distinct classes of methods for modeling the relationship of theoretical constructs. The two classes of methods are integrated in the partial-least-squares approach to structural equation modeling (PLS-SEM), which systematically generates weighted composites and uses…

Distance learning has been adopted as an alternative learning strategy to the face-to-face teaching methodology. It has been largely implemented by many governments worldwide due to the spread of the COVID-19 pandemic and the implication in enforcing lockdown and social distancing. In emergency situations distance learning is referred to as…

Standard setting is arguably one of the most subjective techniques in test development and psychometrics. The decisions when scores are compared to standards, however, are arguably the most consequential outcomes of testing. Providing licensure to practice in a profession has high stake consequences for the public. Denying graduation or forcing…

This Monte Carlo simulation study compares methods to estimate the effects of programs with multiple versions when assignment of individuals to program version is not random. These methods use generalized propensity scores, which are predicted probabilities of receiving a particular level of the treatment conditional on covariates, to remove…

This paper examined the statistical approach adopted by schools to communicate their academic achievement to stakeholders. The paper looked at the most frequently adopted method, the use of percentage scores, and discussed the limitations of its usage. The paper also explored the use of the weighted average of scores and compared it to the use of…

The authors investigate institutional productivity in business and management education (BME) research based on the analysis of 4,464 articles published by 7,210 authors across 17 BME journals over a 10-year period, involving approximately 1,900 schools worldwide. Departing from traditional disciplinary silos, they examine the BME research field…

During the past decade, the NAEP Validity Studies (NVS) Panel has been monitoring, studying, and commenting on potential issues with the validity of the National Assessment of Educational Progress (NAEP) arising from changes that have been brought about by the adoption of rigorous state college and career readiness standards, such as the Common…

Academic departments assign different relative weights to the importance of teaching and research. Those weights are used in making decisions about promotion, tenure, and annual raises. Presumably, raising the teaching weight should encourage faculty to increase time on teaching. Using survey data from U.S. faculty in 1995, 2000, 2005, and 2010,…

When performing a meta-analysis unexplained variation above that predicted by within study variation is usually modeled by a random effect. However, in some cases, this is not sufficient to explain all the variation because of outlier or unusual studies. A previously described method is to define an outlier as a study requiring a higher random…