Construct validity and building validity arguments are some of the main challenges facing the language assessment community. The notion of construct validity and validity arguments arose from research in psychological assessment and developed into the gold standard of validation/validity research in language assessment. At a theoretical level,…

The way we measure values is foundational to their study. This paper explores the empirical findings and theoretical discussions in the literature concerning an essential quality for any measure of values, its validity. We discuss an important debate on validity within the literature concerning the ability of value measures to predict…

The general feeling is that no predictions can be made based on agent-based social simulations. The outcomes of social simulations are based on the behaviors of individuals and their interactions. Behavioral models are always incomplete and often, also incorrect with respect to real behavior and thus the outcomes of agent-based social simulations…

Predictive processing (PP) has been repeatedly presented as a unificatory account of perception, action, and cognition. In this paper, we argue that this is premature: As a unifying theory, PP fails to deliver general, simple, homogeneous, and systematic explanations. By examining its current trajectory of development, we conclude that PP remains…

Selecting courses in an open-curriculum education program is a difficult task for students and academic advisors. Course recommendation systems nowadays can be used to reduce the complexity of this task. To control the recommendation error, we argue that course recommendations need to be provided together with "statistical" confidence.…

There is accumulating evidence that readers continually evaluate the consistency, congruence, and coherence of text by processes of validation. Validation is initiated immediately on stimulus presentation, may proceed nonstrategically, and serves as a criterion for representational updating. However, validation exhibits a variety of deficiencies.…

Hinterecker, Knauff, and Johnson-Laird (2016) compared the adequacy of the probabilistic new paradigm in reasoning with the recent revision of mental models theory (MMT) for explaining a novel class of inferences containing the modal term "possibly." For example, "the door is closed or the window is open or both," therefore,…

Fifteen years ago, Frey and Detterman established that the SAT (and later, with Koenig, the ACT) was substantially correlated with measures of general cognitive ability and could be used as a proxy measure for intelligence (Frey and Detterman, 2004; Koenig, Frey, and Detterman, 2008). Since that finding, replicated many times and cited extensively…

As a personality trait, second language (L2) grit--a combination of perseverance and passion for L2 learning--has recently been proposed as a meaningful predictor of learners' motivational behavior and L2 achievement. The results of a growing body of empirical studies carried out in various L2 contexts have substantiated the power of L2 grit in…

The literature provides multiple measures of diversity along a single demographic dimension, but when it comes to studying the interaction of multiple diversity types (e.g. age, gender, and race), the field of useable measures diminishes. We present the use of decision trees as a machine learning technique to automatically identify the…

This article presents a critique of both transformative learning theory and critical comments on it to date. It argues that transformative learning theory remains substantively the same as its initial exposition, in spite of a raft of problematic contentions voiced against it. The theory is argued here to be conceptually problematic, except at the…

In educational research that calls itself empirical, the relationship between validity and reliability is that of trade-off: the stronger the bases for validity, the weaker the bases for reliability (and vice versa). Validity and reliability are widely regarded as basic criteria for evaluating research; however, there are ethical implications of…

In this paper we argue that simply identifying gateway courses in which a large number of students fail or withdraw and focusing attention on them may not always be the best use of limited resources. No matter what we do, there will always be courses with high D/F/W rates simply because of the nature of their content and the preparation of the…

The construct map is a promising tool for organizing the data standard-setting panelists interpret. The challenge in applying construct maps to standard-setting procedures will be the judicious selection of data to include within this organizing framework. Therefore, this commentary focuses on decisions about what to include in the construct map.…

Teacher observations have become a national education phenomenon, mandated by federal policies and promoted by philanthropists. They are crucial components of teacher evaluation systems that often have high stakes for teachers and school systems, but have sparked little innovation. Recent calls to make teacher evaluations better, faster, and…