Factor analysis is often used to model scales created to measure latent constructs, and internal structure validity evidence is commonly assessed with indices like SRMR, RMSEA, and CFI. These indices are essentially effect size measures and definitive benchmarks regarding which values connote reasonable fit have been elusive. Simulations from the…

Previous research explicates ordinal and disordinal interactions through the concept of the "crossover point." This point is determined via simple regression models of a focal predictor at specific moderator values and signifies the intersection of these models. An interaction effect is labeled as disordinal (or ordinal) when the…

Learning to interpret data in context is an important educational outcome. To assess students' attainment of this outcome, it is necessary to examine the interplay between their contextual and statistical reasoning. We describe a research method designed to do so. The method draws upon Toulmin's (1958, 2003) model of argumentation for the first…

The evidence-based decision-making movement often assumes that once evidence is available (e.g., via the What Works Clearinghouse), decision-makers will integrate it into their practice. Research-practice partnership studies have shown this is not always true. In this paper, we argue that instead of assuming research will be useful and used, we…

Statistical thinking is essential to understanding the nature of scientific results as a consumer. Statistical thinking also facilitates thinking like a scientist. Instead of emphasizing a "correct" procedure for data analysis and its outcome, statistical thinking focuses on the process of data analysis. This article reviews frequentist…

The evidence-based decision-making movement often assumes that once evidence is available (e.g., via the What Works Clearinghouse), decision-makers will integrate it into their practice. Research-practice partnership studies have shown this is not always true. In this paper, we argue that instead of assuming research will be useful and used, we…

Objective: Our master's program in biostatistics requires a qualifying examination (QE). A curriculum review led us to question whether to replace a closed-book format with an open-book one. Our goal was to improve the QE. Methods: This is a case study and commentary, where we describe the evolution of the QE, both in its goals and its content.…

Examples of the impact of statistical theory on assessment practice are provided from the perspective of a statistician trained in theoretical statistics who began to work on assessments. Goodness of fit of item-response models is examined in terms of restricted likelihood-ratio tests and generalized residuals. Minimum discriminant information…

Methods to generate random correlation matrices have been proposed in the literature, but very few instances exist where these correlation matrices are structured or where the statistical properties of the algorithms are known. By relying on the tetrad relation discovered by Spearman and the properties of the beta distribution, an algorithm is…

This paper presents a critical analysis, from the point of view of a statistician, of the methodology used by Hattie in "Visible Learning," and explains why it must absolutely be called pseudoscience. We first discuss what appears to be the intentions of Hattie's approach. Then we describe the major mistakes in "Visible…

Assessment is necessary, but difficult, in statistics education. Multiple choice assessments are common, particularly for research purposes. Open-ended assessments may be more adept at revealing student understanding, but ensuring their validity can be difficult. The study presented here examines differences in student descriptions of histograms…

Currently there are no assessment instruments available for upper-division thermal physics, though several introductory assessments are currently available. Notably missing from these introductory assessment are items targeting statistical mechanics. This leaves a gap in the content that can be assessed by upper-division thermal physics faculty.…

Two-stage exams have gained traction in education as a means of creating collaborative active-learning experiences in the classroom in a manner that advances learning, positively increases student engagement, and reduces test anxiety. Published analyses have focused almost exclusively on the increase in student scores from the first individual…

Formative assessment has long been used to gauge students' understanding of course material prior to taking an exam. With the advent of more advanced technology, only recently have instructors been able to combine formative assessment with a student response system to allow students to respond to questions in real time during class. Previous…

The relevance of the research problem is conditioned by the weak development of measurement, assessment of educational competences at an operational level, at the level of actions, by insufficient applications of psycho-pedagogical theories and methods of mathematical statistics. The aim of the work is to develop through teaching experiments the…