In mixed methods approaches, statistical models are used to identify "nested" cases for intensive, small-n investigation for a range of purposes, including notably the examination of causal mechanisms. This article shows that under a commonsense interpretation of causal effects, large-n models allow no reliable conclusions about effect…

Cross-validation is a common statistical procedure applied to problems that are otherwise computationally intractable. It is often employed to assess the effectiveness of prediction procedures. In this report, cross-validation is discussed in terms of "U"-statistics. This approach permits consideration of the statistical properties of…

Over the last decade, iterated learning studies have provided compelling evidence for the claim that linguistic structure can emerge from non-structured input, through the process of transmission. However, it is unclear whether individuals differ in their tendency to add structure, an issue with implications for understanding who are the agents of…

Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. To aid the development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (1) in the translation of real-world goals to goals on a particular set of training data, (2) in the…

Traditional school laboratory exercises on a system of moving objects connected by strings involve deriving expressions for the system acceleration, a = (?F)/m, and sketching a graph of acceleration vs. force. While being in the form of rational functions, these expressions present great opportunities for broadening the scope of the analysis by…

The author of this commentary argues that physical scientists are attempting to advance knowledge in the so-called hard sciences, whereas education researchers are laboring to increase knowledge and understanding in an "extremely hard" but softer domain. Drawing on the work of Popper and Dewey, this commentary highlights the relative…

This chapter describes a concrete process that stakeholders can use to make predictions about the future performance of programs in local contexts. Within the field of evaluation, the discussion of validity as it relates to outcome evaluation seems to be focused largely on questions of internal validity (Did it work?) with less emphasis on…

This primer should provide the reader with a deeper understanding of the concept of test validity and will present the recent available validity evidence on the relationship between SAT® scores and important college outcomes. In addition, the content examined on the SAT will be discussed as well as the fundamental attention paid to the fairness of…

A wealth of empirical evidence has now accumulated concerning animals' categorizing photographs of real-world objects. Although these complex stimuli have the advantage of fostering rapid category learning, they are difficult to manipulate experimentally and to represent in formal models of behavior. We present a solution to the representation…