Although originally conceived of as a marriage of direct behavioral observation and indirect behavior rating scales, recent research has indicated that Direct Behavior Ratings (DBRs) are affected by rater idiosyncrasies (rater effects) similar to other indirect forms of behavioral assessment. Most of this research has been conducted using…

Computational thinking (CT) is a fundamental ability for learners in today's society. Although CT assessments and interventions have been studied widely, little is known about CT predictions. This study predicted students' CT achievement in the ICILS 2018 using five machine learning models. These models were trained on the data from five European…

Current student-centred, multilingual, active teaching methodologies require that teachers have continuous access to texts that are adequate in terms of topic and language competence. However, the task of finding appropriate materials is arduous and time consuming for teachers. To build on automatic readability assessment research that could help…

Educational reform priorities such as emphasis on quantitative modelling (QM) have positioned undergraduate biology instructors as designers of QM experiences to engage students in authentic science practices that support the development of data-driven and evidence-based reasoning. Yet, little is known about how biology instructors adapt to the…

Researchers across many fields have called for greater attention to heterogeneity of treatment effects--shifting focus from the average effect to variation in effects between different treatments, studies, or subgroups. True heterogeneity is important, but many reports of heterogeneity have proved to be false, non-replicable, or exaggerated. In…

Attrition in longitudinal studies is a major threat to the representativeness of the data and the generalizability of the findings. Typical approaches to address systematic nonresponse are either expensive and unsatisfactory (e.g., oversampling) or rely on the unrealistic assumption of data missing at random (e.g., multiple imputation). Thus,…

Over the past 50 years, scientists have made amazing discoveries about the origins of human language acquisition. Central to this field of study is the process by which infants' perceptual sensitivities gradually align with native language structure, known as "perceptual narrowing." Perceptual narrowing offers a theoretical account of…

Experimental studies of educational interventions are rarely based on representative samples of the target population. This simulation study tests two formal sampling strategies for selecting districts and schools from within strata when they may not agree to participate if selected: (1) balanced selection of the most typical district or school…

The study of gender identity is becoming increasingly important to the field of higher education as younger generations of students enter their undergraduate programs. "As the number of college students identifying as transgender," gender nonconforming, or another gender identity, increases, "so too does the need to understand their…

Mathematical reasoning, a fundamental aspect of human cognition, poses significant challenges for artificial intelligence (AI) systems. Despite recent advancements in natural language processing (NLP) and large language models (LLMs), AI's ability to replicate human-like reasoning, generalization, and efficiency remains an ongoing research…

Much of learning analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects, such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors…

This article extends multivariate generalizability theory (MGT) to tests with different random-effects designs for each level of a fixed facet. There are numerous situations in which the design of a test and the resulting data structure are not definable by a single design. One example is mixed-format tests that are composed of multiple-choice and…

This paper aims to unsettle assumptions of generalizability and representativeness in quantitative research using queer framings and positionalities. We argue that generalizability and representativeness are tools of supremacist dominance that reinforce harmful and essentialist categories of identities for the false purpose of statistical…

The costs of an objective structured clinical examination (OSCE) are of concern to health profession educators globally. As OSCEs are usually designed under generalizability theory (G-theory) framework, this article proposes a machine-learning-based approach to optimize the costs, while maintaining the minimum required generalizability…

A conceptual confusion of "theory" exists in South African Open Distance and e-Learning (ODeL) research in which the concept is used with borderline, related, contrary, invented, or illegitimate concepts such as a model, approach, construct, hypothesis, theoretical framework, or conceptual framework. As a result, some researchers choose…