Although data modeling, the employment of statistical reasoning for the purpose of investigating questions about the world, is central to both mathematics and science, it is rarely emphasized in K-16 instruction. The current work focuses on developing thinking about data modeling with undergraduates in general and preservice teachers in…

Best linear prediction (BLP) and penalized best linear prediction (PBLP) are techniques for combining sources of information to produce task scores, section scores, and composite test scores. The report examines issues to consider in operational implementation of BLP and PBLP in testing programs administered by ETS [Educational Testing Service].

When estimating causal effects, unmeasured confounding and model misspecification are both potential sources of bias. We propose a method to simultaneously address both issues in the form of a semi-parametric sensitivity analysis. In particular, our approach incorporates Bayesian Additive Regression Trees into a two-parameter sensitivity analysis…

Propensity score analysis (PSA) is a methodological technique which may correct for selection bias in a quasi-experiment by modeling the selection process using observed covariates. Because logistic regression is well understood by researchers in a variety of fields and easy to implement in a number of popular software packages, it has…

Product assessment is widely applied in creative studies, typically as an important dependent measure. Within this context, this study had 2 purposes. First, the focus of this research was on methods for investigating possible rater effects, an issue that has not received a great deal of attention in past creativity studies. Second, the…

A paradoxical implication of Kraemer's expression for the large-sample standard error of Brogden's form of the biserial correlation is identified, and a new expression is given which does not imply the paradox. However, numerical evidence is presented which calls into question the correctness of the expression. (Author)

Used simulation to study two different approaches to latent variable interaction modeling with continuous observed variables: (1) a LISREL 8.30 program and (2) data analysis through PRELIS2 and SIMPLIS programs. Results show that parameter estimation was similar but standard errors were different. Discusses differences in ease of implementation.…

Asymptotic distribution theory of Brogden's form of biserial correlation coefficient is derived and large sample estimates of its standard error obtained. Its relative efficiency to the biserial correlation coefficient is examined. Recommendations for choice of estimator of biserial correlation are presented. (Author/JKS)

Previous methods for estimating the conditional standard error of measurement (CSEM) at specific score or ability levels are critically discussed, and a brief summary of prior empirical results is given. A new method is developed which avoids theoretical problems inherent in some prior methods, is easy to implement, and estimates not only a…

Simulated data based on five test score distributions indicate that a slight modification of the asymptotic normal theory for the estimation of the p and kappa indices in mastery testing will provide results which are in close agreement with those based on small samples from the beta-binomial distribution. (Author/BW)

This paper considers some appropriate and inappropriate uses of coefficient kappa and alternative kappa-like statistics. Discussion is restricted to the descriptive characteristics of these statistics for measuring agreement with categorical data in studies of reliability and validity. (Author)

Under the assumption of normalcy, a formula is derived for the reliability of the maximum score. It is shown that the maximum score is more reliable than each of the single observations but less reliable than their composite score. (Author/LMO)

Both the binomial and beta-binomial models are applied to various problems occurring in mental test theory. The paper reviews and critiques these models. The emphasis is on the extensions of the models that have been proposed in recent years, and that might not be familiar to many educators. (Author)

Huynh (1976) suggested a method of approximating the reliability coefficient of a mastery test. The present study examines the accuracy of Huynh's approximation and also describes a computationally simpler approximation which appears to be generally more accurate than the former. (Author/RL)

Unreliable profiles impose the difficulty that ordinal and interval relations among the individual's scores become uncertain or unstable. A profile reliability coefficient is derived to estimate the relative expected extent of this ordinal and interval "inversion" for any profile of K measures. (Author/DWH)