Generative AI tools (GenAI) are increasingly used for academic tasks, including qualitative data analysis for the Scholarship of Teaching and Learning (SoTL). In our practice as academic developers, we are frequently asked for advice on whether this use for GenAI is reliable, valid, and ethical. Since this is a new field, we have not been able to…

Research on classroom motivational climates and microclimates--students' shared and idiosyncratic perceptions of motivational classroom features--demonstrates their importance for fostering adaptive motivational and achievement-related outcomes. However, a lack of coherent theoretical guidance about the nature of students' classroom climate…

Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most…

Assessing student achievement over multiple years is complicated by students' memberships in shifting upper-level nesting structures. These structures are manifested in (1) annual matriculation to different classrooms and (2) mobility between schools. Failure to model these shifting upper-level nesting structures may bias the inferences…

It is of critical importance to be aware of the historical discrimination embedded in the data and to consider a fairness measure to reduce bias throughout the predictive modeling pipeline. Given various notions of fairness defined in the literature, investigating the correlation and interaction among metrics is vital for addressing unfairness.…

Traditional validation approaches fail to account for the ways oppressive systems (e.g. racism, radical nationalism) impact the test design and development process. To disrupt this legacy of white supremacy, we illustrate how justice-oriented, antiracist validation (JAV) framework can be applied to construct articulation and validation, data…

The designs of many student-facing learning analytics (SFLA) dashboards are insufficiently informed by educational research and lack rigorous evaluation in authentic learning contexts, including during remote laboratory practical work. In this article, we present and evaluate an SFLA dashboard designed using the principles of formative assessment…

These are the appendixes for the report, "Strengthening the Pennsylvania School Climate Survey to Inform School Decisionmaking." This study analyzed Pennsylvania School Climate Survey data from students and staff in the 2021/22 school year to assess the validity and reliability of the elementary school student version of the survey;…

Higher education institutes experience lower success rates in online learning environments compared to traditional learning. Students' engagement within the learning management system (LMS) is one of the main factors affecting students' academic performance and retention. This quantitative correlational-predictive study examined if, and to what…

We report the results of a convergent validity study examining the externalizing subscale (SRSS-E5, five items) of the adapted Student Risk Screening Scale for Internalizing and Externalizing (SRSS-IE 9) with the externalizing subscale of the Teacher Report Form (TRF) with two samples of K-12 students. Results of logistic regression and receiver…

Purpose: The purpose of this study was to ascertain the relationship between EFL teachers' perception of the intended and unintended consequences of formative assessment (FA) decisions and their sense of self-efficacy and anxiety toward data-driven decision-making (DDDM). Design/methodology/approach: A correlational research design and…

Reasoning for complexity is a fundamental competency in these complex times for solutions to social problems and decision-making. The purpose of this paper is to demonstrate the validity and reliability of the eComplexity instrument by presenting its psychometric properties. The instrument consists of a Likert-type scale questionnaire designed to…

Digital literacy is essential for scientific literacy in a digital world. Although the NGSS Practices include many activities that require digital literacy, most studies have examined digital literacy from a generic perspective rather than a curricular context. This study aimed to develop a self-report tool to measure elements of digital literacy…