The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

This research aimed to develop and validate a rubric to assess Artificial Intelligence (AI) chatbots' effectiveness in accomplishing tasks, particularly within educational contexts. Given the rapidly growing integration of AI in various sectors, including education, a systematic and robust tool for evaluating AI chatbot performance is essential.…

Computation competence (CC) in simple addition and subtraction using non-counting (NC) strategies is an important learning objective in Grade 1 mathematics but many children, especially low achievers in mathematics, struggle to acquire these skills. To provide these students with the support they need, it is important to have valid and reliable…

The mode of assessment is one of the most important factors influencing learning. E-assessment usually includes only checking the final answer, thus limiting teacher's ability to check the complete solution, and it does not allow inclusion of math proofs problems that constitute an important part of math content. The e-assessment module of Halomda…

Research Purpose and Contribution: The study aimed to construct an evaluation framework for assessing pupils' computational thinking (CT) during classroom learning problem solving. As a self-report evaluation scale for pupils, this evaluation framework further enriched the CT assessment instruments for pupils and provided a specialized instrument…

This study was carried out to develop a test to assess algorithmic thinking skills. To this end, the twelve steps suggested by Downing (2006) were adopted. Throughout the test development, 24 middle school sixth-grade students and eight experts in different areas took part as needed in the tasks on the project. The test was given to 252 students…

The Programme for the International Assessment of Adult Competencies (PIAAC) assesses key skills in literacy, numeracy and problem-solving in technology-rich environments, as well as their relationship to other social outcomes for adults. PIAAC's developers claimed to better account for adults' literacy practices than earlier international studies…

This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather…

Purpose: To evaluate the validity, reliability, and sensitivity of the novel Means-End Problem-Solving Assessment Tool (MEPSAT). Methods: Children with typical development and those with motor delay were assessed throughout the first 2 years of life using the MEPSAT. MEPSAT scores were validated against the cognitive and motor subscales of the…

Science learning in the 21st century should develop the abilities of the students to find solutions to the problems. This task requires a set of skills called Higher Order Thinking Skills (HOTS) and innovative assessment should be designed to gauge the acquisition of these skills. Thus, this study aimed to create an innovative HOTS-based…

Background: The shortage of skilled workers choosing STEM (Science, Technology, Engineering, and Math) careers in the USA and worldwide has fueled a movement towards STEAM, in which the "A" addresses the arts and humanities. STEAM education has been proposed as a way to offer relevant problems to solve while drawing on creative and…

Educators' Social-Emotional Learning (SEL) is crucial for fostering positive, supportive, and effective learning environments. This study seeks to improve SEL assessment among educators by addressing limitations of the previous EduSEL questionnaire. Study 1 established convergent validity by comparing EduSEL with a validated SEL questionnaire.…

Studies have acknowledged computational thinking (CT) as an efficient approach for problem-solving particularly required in digital workplaces. This research aims to identify indicators for a holistic CT assessment instrument for undergraduate students. A three-round fuzzy Delphi study has been conducted to gain comprehensive opinions and…

This paper presents evidence of the score reliability, factor structure, criterion-related validity, and measurement equivalence of a web-based assessment of several important social and emotional competencies for children in fourth through sixth grades. The assessment, SELweb LE (Late Elementary), is designed to measure children's understanding…

Assessing complex constructs such as those discussed under the umbrella of 21st century constructs highlights the need for a principled assessment design and validation approach. In our discussion, we made a case for three considerations: (a) taking construct complexity into account across various stages of assessment development such as the…