This paper presents a framework for utilizing Student Evaluations of Teaching (SET) in faculty evaluations. Recognizing the ongoing debate about the validity of SET as a measure of teaching effectiveness, the authors agree with scholars who propose viewing SET as a tool for gauging 'student perceptions of learning'. They present a method that…

The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Learning to interpret data in context is an important educational outcome. To assess students' attainment of this outcome, it is necessary to examine the interplay between their contextual and statistical reasoning. We describe a research method designed to do so. The method draws upon Toulmin's (1958, 2003) model of argumentation for the first…

In the last few years, Qualitative Comparative Analysis (QCA) has become one of the most important data analysis methods in comparative research. According to the guidelines of this method, there are certain steps that a researcher needs to follow, before causally analyzing the data for necessary and sufficient conditions. One of these steps is…

High dropout rate exists universally in massive open online courses (MOOCs) due to the separation of teachers and learners in space and time. Dropout prediction using the machine learning method is an extremely important prerequisite to identify potential at-risk learners to improve learning. It has attracted much attention and there have emerged…

This research brief describes recent methodological initiatives with the Public Libraries Survey. It describes how starting with the 2022 data, Institute of Museum and Library Services (IMLS) updated geographic identifiers to better align Census Bureau geography types with the library's legal service area, with the goal of enabling data users to…

Gas Turbine Engines (GTE) have the highest power-to-weight ratio among Internal Combustion Engines (ICE). Its modularity and ability to utilize various types of fuel make it highly recommended in power plants, naval transportation, and, of course, the most equipped in aviation. The lack of GTEs' real data is increasing a recognized need for…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Since the No Child Left Behind Act (No Child Left Behind [NCLB], 2001) was enacted, the Bookmark method has been used in many state standard setting studies (Karantonis and Sireci; Zieky, Perie, and Livingston). The purpose of the current study is to evaluate the criticism that when panelists are presented with data during the Bookmark standard…

Monitoring applications (MAs) use digital and online tools to collect and track data on student behavior, and they have become increasingly popular among schools. Empirical research on these complex surveillance platforms is scant, and little is known about the efficacy or impact that they have on students. This study used a multi-method…

To provide personalized support on educational platforms, it is crucial to model the evolution of students' knowledge states. Knowledge tracing is one of the most popular technologies for this purpose, and deep learning-based methods have achieved state-of-the-art performance. Compared to classical models, such as Bayesian knowledge tracing, which…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Despite many calls, there is little research addressing teachers' knowledge of reading assessments and how they utilize assessments for reading instruction. Therefore, the current research developed and validated a reliable measure of teachers' perceptions and knowledge of reading assessments, called the "Perceptions and Knowledge of…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…