In this article, we propose using the Bayes factors (BF) to evaluate person fit in item response theory models under the framework of Bayesian evaluation of an informative diagnostic hypothesis. We first discuss the theoretical foundation for this application and how to analyze person fit using BF. To demonstrate the feasibility of this approach,…

We present a multiple-choice test, the Montana State University Formal Reasoning Test (FORT), to assess college students' scientific reasoning ability. The test defines scientific reasoning to be equivalent to formal operational reasoning. It contains 20 questions divided evenly among five types of problems: control of variables, hypothesis…

Making causal inferences from a quasi-experiment is difficult. Sensitivity analysis approaches to address hidden selection bias thus have gained popularity. This study serves as an introduction to a simple but practical form of sensitivity analysis using Monte Carlo simulation procedures. We examine estimated treatment effects for a school-based…

This paper introduces a dataset and associated analysis of the scores of National Football League (NFL) games over the 2012, 2013, and first five weeks of the 2014 season. In the face of current media attention to "lopsided" scores in Thursday night games in the early part of the 2014 season, t-test results indicate no statistically…

In this brief report, we examine whether students' (N = 230) willingness to help individuals in distress (altruism) would be augmented after viewing Secure Attachment Messages (SAM) during lecture in a college racism course. Students were presented with SAM in alternating weeks as part of the PowerPoint presentation slides. In each of the weeks,…

Recent research suggests that the use of student search services is an effective part of a college's student marketing and recruitment strategy. What is not clear, however, is whether participating in a search service is an effective part of a student's college search strategy. To address this question, we exploit a recent change in the choice…

In the literature, the potential efficacy of the gamification of education has been demonstrated. The aim of this study was to explore the influence of applying gamification techniques to increase student engagement in learning. The quasi-experimental nonequivalent-control group design was used with 577 undergraduate students from six classes. The…

Differential bundle functioning (DBF) analyses were conducted to determine whether seventh and eighth grade second language learners (SLLs) had lower probabilities of answering bundles of math word problems correctly that had heavy language demands, when compared to non-SLLs of equal math proficiency. Math word problems on each of four test forms…

This article presents the original model of the computer adaptive testing and grade formation, based on scientifically recognized theories. The base of the model is a personalized algorithm for selection of questions depending on the accuracy of the answer to the previous question. The test is divided into three basic levels of difficulty, and the…

This dissertation is composed of three distinct chapters, each of which addresses issues of estimating treatment effects. The first chapter empirically tests the Value-Added (VA) model using school lotteries. The second chapter, co-authored with Michael Wood, considers properties of inverse probability weighting (IPW) in simple treatment effect…

Research Findings: This study used data from a national sample to examine differences in school outcomes at the end of kindergarten between Head Start children who attended full-day and half-day programs. Propensity scores were used to match children who experienced different intensities of the program on a series of demographic characteristics in…

We compared 5 different statistics (i.e., G index, gamma, "d'", sensitivity, specificity) used in the social sciences and medical diagnosis literatures to assess calibration accuracy in order to examine the relationship among them and to explore whether one statistic provided a best fitting general measure of accuracy. College…

This study examined the effectiveness of two methods of increasing student learning from posterior probability lessons: diagrams and questioning while reading. Undergraduate students (N = 245) read a lesson in one of three diagram conditions and one of three questioning-while-reading conditions (embedded questions, elaboration interrogations, and…

Recently, the authors have been exploring the use of propensity score methods for developing evidence of program impact. Specifically, they have been developing evidence (after one year of implementation) of the effects of the Math Science Partnership in New York City ("MSPinNYC2") on high school students' achievement--both in terms of…

Often researchers face situations where comparative studies between two or more programs are necessary to make causal inferences for informed policy decision-making. Experimental designs employing randomization provide the strongest evidence for causal inferences. However, many pragmatic and ethical challenges may preclude the use of randomized…