Profile analysis is one of the main tools for studying whether differential item functioning can be related to specific features of test items. While relevant, profile analysis in its current form has two restrictions that limit its usefulness in practice: It assumes that all test items have equal discrimination parameters, and it does not test…

This article explores the potential of artificial intelligence (AI) for academic advising. Specifically, it examines how AI-powered machine interpretation and data analysis can be used to deliver advising services anytime, anywhere. This system would eliminate the need for students to physically meet with advisors and could answer their…

Computational thinking (CT) is a fundamental ability for learners in today's society. Although CT assessments and interventions have been studied widely, little is known about CT predictions. This study predicted students' CT achievement in the ICILS 2018 using five machine learning models. These models were trained on the data from five European…

The costs of an objective structured clinical examination (OSCE) are of concern to health profession educators globally. As OSCEs are usually designed under generalizability theory (G-theory) framework, this article proposes a machine-learning-based approach to optimize the costs, while maintaining the minimum required generalizability…

A conceptual confusion of "theory" exists in South African Open Distance and e-Learning (ODeL) research in which the concept is used with borderline, related, contrary, invented, or illegitimate concepts such as a model, approach, construct, hypothesis, theoretical framework, or conceptual framework. As a result, some researchers choose…

Many high-stakes examinations in the United Kingdom (UK) use both constructed-response items and selected-response items. We need to evaluate the inter-rater reliability for constructed-response items that are scored by humans. While there are a variety of methods for evaluating rater consistency across ratings in the psychometric literature, we…

This study aims to compare the G and Phi coefficients as estimated by D studies for a measurement tool with the G and Phi coefficients obtained from real cases in which items of differing difficulty levels were added and also to determine the conditions under which the D studies estimated reliability coefficients closer to reality. The study group…

This article serves as an introduction to the Special Issue on "Decolonizing and Diversifying Research in Cognitive Development." The Special Issue comprises six articles: two articles are empirical articles that focus on executive function development in under-represented environments, two articles address barriers pathways toward…

Writing assessments often consist of students responding to multiple prompts, which are judged by more than one rater. To establish the reliability of these assessments, there exist different methods to disentangle variation due to prompts and raters, including classical test theory, Many Facet Rasch Measurement (MFRM), and Generalizability Theory…

It has become a critical question what the reliability level would be when open-ended questions are used in large-scale selection tests. One of the aims of the present study is to determine what the reliability would be in the event that the answers given by test-takers are scored by experts when open-ended short answer questions are used in…

Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school's…

Integrating basic principles, values, and practices of education for sustainability in early childhood education (ECE) is important for developing and developed countries. In this context, there is a need to identify and evaluate indicators that measure sustainable practices in ECE programs. This study was conducted in 18 preschools in Turkey to…

Purpose: This study intended to examine the generalizability and reliability of essay ratings within the scope of the generalizability (G) theory. Specifically, the effect of raters on the generalizability and reliability of students' essay ratings was examined. Furthermore, variations of the generalizability and reliability coefficients with…

One of the important theories in education and psychology is Generalizability (G) Theory and various properties distinguish it from the other measurement theories. To better understand methodological trends of G theory, a thematic content analysis was conducted. This study analyzes the studies using generalizability theory in the field of…

Conscientiousness and interest are well-known predictors of academic effort and achievement. As hypothesized by the Conscientiousness × Interest Compensation (CONIC) model, conscientiousness and interest can (partly) compensate for each other, leading to (comparatively) high effort if either conscientiousness or interest is high. The present…