Although it is commonly believed that automation aids human decision-making, conflicting evidence raises questions about whether individuals would gain greater advantages from automation in difficult tasks. Our study examines the combined influence of task difficulty and automation reliability on aided decision-making. We assessed decision…

The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting-edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely,…

The procedures of carrying out factorial invariance to validate a construct were well developed to ensure the reliability of the construct that can be used across groups for comparison and analysis, yet mainly restricted to the frequentist approach. This motivates an update to incorporate the growing Bayesian approach for carrying out the Bayesian…

Purpose: Artificial intelligence (AI) is more capable and accessible than ever before. But what does this mean for clinical practice? How can speech-language clinicians evaluate the efficacy, validity, and reliability of AI and machine learning tools for automating assessment and treatment? How can speech-language clinicians ethically use these…

As generative artificial intelligence tools like ChatGPT become increasingly integrated into educational environments, understanding their impact on critical thinking is crucial. Despite growing concerns about AI's potential to diminish students' independent reasoning, there is a lack of research tools specifically designed to evaluate students'…

This study investigates the validity and reliability of generative large language models (LLMs), specifically ChatGPT and Google's Bard, in grading student essays in higher education based on an analytical grading rubric. A total of 15 experienced English as a foreign language (EFL) instructors and two LLMs were asked to evaluate three student…

Individuals are controlled by tests in every advanced society when they want to be admitted in educational courses, to proceed from one stage to the next, or to be given a certificate (Shohamy, 2001b). Accordingly, the present study was carried out to construct and validate educational, social, and psychological consequences questionnaires of…

This study aimed to develop a valid and reliable scale to measure individuals' and organizations' attitudes toward the use of ChatGPT, emphasizing the necessity for organizations to adapt to rapidly evolving information and technology environments. The methodology consisted of three stages. In the first stage, a 13-item draft scale was…

The purpose of this paper is to investigate the descriptive and evaluative findings of articles conducted in educational contexts on entrepreneurship education in the Web of Science (WoS) database. For this purpose, the bibliometric analysis method was used in this study. This systematic literature review examined 352 scientific articles published…

As technology advances, AI, like ChatGPT, has become a pivotal tool in improving educational practices, particularly in special education (SE). These tools support the fourth Sustainable Development Goal (SDG), which concerns quality education for all, and the tenth SDG addresses reducing disparities. The purpose of this study is to explore the…

To address the emerging trend of language learning with Artificial Intelligence (AI), this study explored junior and senior high school students' behavioral intentions to use AI in second language (L2) learning, and the roles of related technological, social, and motivational factors. An eight-factor survey was constructed using a 5-point Likert…

Multilevel modeling (MLM) is a statistical technique for analyzing clustered data. Despite its long history, the technique and accompanying computer programs are rapidly evolving. Given the complexity of multilevel models, it is crucial for researchers to provide complete and transparent descriptions of the data, statistical analyses, and results.…

Accurately grading open-ended assignments in large or massive open online courses is nontrivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To date, no work has leveraged sentiment analysis in the peer-review process to inform or validate grades or utilized aspect extraction to…

Introduction: This study examined the concurrent validity and inter-pair reliability of the proximity detection function on Bluetooth-enabled accelerometers across manufacturer-specified ranges. If valid, this method could be used for objectively measuring shared physical activity participation. Method: Thirty-six dyads aged 21.6 (2.1) years wore…

The present research paper introduces a translation evaluation method called Calibrated Parsing Items Evaluation (CPIE hereafter). This evaluation method maximizes translators' performance through identifying the parsing items with an optimal p-docimology and d-index (item discrimination). This method checks all the possible parses (annotations)…