Generative AI tools (GenAI) are increasingly used for academic tasks, including qualitative data analysis for the Scholarship of Teaching and Learning (SoTL). In our practice as academic developers, we are frequently asked for advice on whether this use for GenAI is reliable, valid, and ethical. Since this is a new field, we have not been able to…

Background: Peer assessment has played an important role in large-scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop…

This work aims to elaborate a governance composite index for research laboratories in public university. This index is composed by an indicator of responsible liberalism associating effectiveness and ethics, to which are added an organizational management indicator and a strategic management indicator. To achieve the above aim, several methods are…

Ensuring consistent and reliable scoring is paramount in education, especially in performance-based assessments. This study delves into the critical issue of marking consistency, focusing on speaking proficiency tests in English language learning, which often face greater reliability challenges. While existing literature has explored various…

Video data analysis (VDA) represents an important methodological framework for contemporary research approaches to the myriad of footage available from cameras, devices, and phones. Footage from police body-worn cameras (BWCs) is anticipated to be a widely available platform for social science researchers to scrutinize the interactions between…

Systematic reviews are vital for guiding practice, research and policy, although they are often slow and labour-intensive. Large language models (LLMs) could speed up and automate systematic reviews, but their performance in such tasks has yet to be comprehensively evaluated against humans, and no study has tested Generative Pre-Trained…

Although parallel analysis has been found to be an accurate method for determining the number of factors in many conditions with complete data, its application under missing data is limited. The existing literature recommends that, after using an appropriate multiple imputation method, researchers either apply parallel analysis to every imputed…

Analysis of data from single-case intervention studies commonly involves visual analysis. Previous research indicates that visual analysis may suffer from low reliability and unpromising error rates. We investigated the reliability and validity of visual analysis and explored to what extent data trends affect judgments. We administered a…

In recognition of the capability of text-mining models to quantify aspects of language use, some creativity researchers have adopted text-mining models as a mechanism to objectively and efficiently score the Originality of open-ended responses to verbal divergent thinking tasks. With the increasing use of text-mining models in divergent thinking…

Conducting valid and reliable empirical research in the prevention sciences is an inherently difficult and challenging task. Chief among these is the need to obtain numerical scores of underlying theoretical constructs for use in subsequent analysis. This challenge is further exacerbated by the increasingly common need to consider multiple…

These are the appendixes for the report, "Strengthening the Pennsylvania School Climate Survey to Inform School Decisionmaking." This study analyzed Pennsylvania School Climate Survey data from students and staff in the 2021/22 school year to assess the validity and reliability of the elementary school student version of the survey;…

The purpose of this study is to examine the abstracts in 6th International Preschool Education Congress abstract book in terms of research subject, method, model, sample type, data collection tools, data analysis techniques, validity and reliability. This study is a qualitative study and was conducted using document analysis, which is one of the…

Background and Aims: The fastest growing group of students with disabilities are those with Autism Spectrum Disorder (ASD). States annually report on post-high school outcomes (PSO) of exited students. This study sought to fill two gaps in the literature related to PSO for exited high-school students with ASD and the use of state data and…

It is seen that the number of studies dealing with the values in children's literary products has been increased. Although there are many reviews on the studies concerning with value education, these reviews do not provide some significant details about these studies. This study has provided a content analysis of the studies on the values in…

Background: Systematic reviewers often encounter incomplete or missing data, and the information desired may be difficult to obtain from a study author. Thus, systematic reviewers may have to resort to estimating data from figures with little or no raw data in a study's corresponding text or tables. Methods: We discuss a case study in which…