Classroom observational protocols, in which raters observe and score the quality of teachers' instructional practices, are often used to evaluate teachers for consequential purposes despite evidence that scores from such protocols are frequently driven by factors, such as rater and temporal effects, that have little to do with teacher quality. In…

Structural equation modeling (SEM) is a widely used technique for studies involving latent constructs. While covariance-based SEM (CB-SEM) permits estimating the regression relationship among latent constructs, the parameters governing this relationship do not apply to that among the scored values of the constructs, which are needed for…

This three-article dissertation aims to address three methodological challenges to ensure comparability in educational research, including scale linking, test equating, and propensity score (PS) weighting. The first study intends to improve test scale comparability by evaluating the effect of six missing data handling approaches, including…

Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student…

Reporting confidence intervals with test scores helps test users make important decisions about examinees by providing information about the precision of test scores. Although a variety of estimation procedures based on the binomial error model are available for computing intervals for test scores, these procedures assume that items are randomly…

Background: We adopted a theoretical framework that the acquisition of a scientific concept comprises the development of connections among conceptual elements associated with a scientific term within a mental semantic network. Given this framework, the hypothesis that the surrounding words connected with a scientific term are relevant to the…

World university rankings use the weight-and-sum approach to process data. Although this seems to pass the common sense test, it has statistical problems. In recent years, seven such problems have been uncovered: spurious precision, weight discrepancies, assumed mutual compensation, indictor redundancy, inter-system discrepancy, negligence of…

Most massive open online courses (MOOC) use simple schemes for aggregating peer grades, taking the mean or the median, or compute weights from information other than the instructor's opinion about the students' knowledge. To reduce the difference between the instructor and students' aggregated scores, some proposals compute specific weights to…

During the past decade, the NAEP Validity Studies (NVS) Panel has been monitoring, studying, and commenting on potential issues with the validity of the National Assessment of Educational Progress (NAEP) arising from changes that have been brought about by the adoption of rigorous state college and career readiness standards, such as the Common…

Knowledge of dental anatomy is of great importance in the practice of dentistry, especially in oral rehabilitation, because without this knowledge, professional practice is not possible. Dental carving plays a major role in training dental students as it develops their manual dexterity. This randomized controlled trial aimed to evaluate the…

A little-known theorem, a generalization of Pythagoras's theorem, due to Pappus, is used to present a geometric explanation of various definitions of the contribution of component tests to their composite. I show that an unambiguous definition of the unique contribution of a component to the composite score variance is present if and only if the…

This study investigates technical efficiency and productivity for Swedish higher education institutions (HEIs). One identified problem in previous research concerns adjusting efficiency scores for input quality. This problem is avoided using grades from upper-secondary schools. A second problem concerns heterogeneity with respect to subjects and…

Bayesian alternatives to frequentist propensity score approaches have recently been proposed. However, few studies have investigated their covariate balancing properties. This article compares a recently developed two-step Bayesian propensity score approach to the frequentist approach with respect to covariate balance. The effects of different…

International student enrollment has experienced dramatic increases on U.S. campuses. Using a national dataset, the study explores and compares international and domestic students' incoming and post-training levels of motivation to lead, leadership self-efficacy, and leadership skill using inverse-probability weighting of propensity scores to…

Objective: This article brings the discussions on incorporating perceived importance across study areas into the study of client satisfaction and cautions the use of multiplicative scores (multiplying satisfaction and importance scores) as a weighting method. An alternative weighting method is provided. Method: Analyze data from a client…