In this article, Julia So discusses the purpose of program assessment, four common missteps of program assessment and reporting, and how to prevent them. The four common missteps of program assessment and reporting she has observed are: (1) unclear or ambiguous program goals; (2) measurement error of program goals and outcomes; (3) incorrect unit…

This study compares how the generative AI (GenAI) large language model (LLM) ChatGPT performs in grading university exams compared to human teachers. Aspects investigated include consistency, large discrepancies and length of answer. Implications for higher education, including the role of teachers and ethics, are also discussed. Three…

The primary purpose of this study is to validate a student engagement measure for its use in evaluation of a learning assistant (LA) program. A series of psychometric evaluations were made for both the original scale of Higher Education Student Engagement Scale (HESES) and its adapted version designed to be used in gauging the effectiveness of…

Previous research on student thinking about experimental measurement and uncertainty has primarily focused on students' procedural reasoning: Given some data, what should students calculate or do next? This approach, however, cannot tell us what beliefs or conceptual understanding leads to students' procedural decisions. To explore this…

The Q-matrix played a key role in implementations of diagnostic classification models (DCMs) or cognitive diagnostic models (CDMs) -- a family of psychometric models that are gaining attention in providing diagnostic information on students' mastery of cognitive attributes or skills. Using two Monte Carlo simulation studies, this dissertation…

This article focuses on rater severity and consistency and their relation to different types of rater experience over a long period of time. The article is based on longitudinal data collected from 2009 to 2019 from the second language Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. The study investigated…

We illustrate how to analyze complete generalizability theory (GT) designs using structural equation modeling software ("lavaan" in R), compare results to those obtained from numerous ANOVA-based packages, and apply those results in practical ways using data obtained from a large sample of respondents, who completed the Self-Perception…

Variability is underemphasized in domains such as engineering. Statistics and data science education research offers a variety of frameworks for understanding variability, but new frameworks for domain applications are necessary. This study investigated the professional practices of working engineers to develop such a framework. The Neglected,…

Scientific research is vital for student's education, fostering critical thinking, problem-solving skills, and deepening subject knowledge. To assess students' attitudes towards research, the attitude towards research scale was developed (EACIN). This study addresses three gaps regarding this instrument: inconsistent latent structure, lack of…

Intelligence tests are used in a variety of settings such as schools, clinics, and courts to assess the intellectual capacity of individuals of all ages. Intelligence tests are used to make high-stakes decisions such as special education placement, employment, eligibility for social security services, and determination of the death penalty.…

Students are assessed by coursework and/or exams, all of which are marked by assessors (markers). Student and marker performances are then subject to end-of-session board of examiner handling and analysis. This occurs annually and is the basis for evaluating students but also the wider learning and teaching efficiency of an academic institution.…

The overall aim was to examine effects of differences in group ability and features of the anchor test form on equating bias and the standard error of equating (SEE) using both real and simulated data. Chained kernel equating, Postratification kernel equating, and Circle-arc equating were studied. A college admissions test with four different…

Examinations of the internal structure of the Depression, Anxiety, and Stress Scale-21 (DASS-21) have yielded inconsistent conclusions within and across cultural contexts. This study examined the dimensionality and reliability of the DASS-21 across three theoretically plausible factor structures (i.e., unidimensional, oblique three-factor, and…

Student well-being has gradually become a topic of interest in higher education, and the accurate, valid, and reliable measure of well-being constructs is crucial in the South African context. This study examined item bias and configural, metric and scalar invariance of the Satisfaction with Life Scale (SWLS) for South African first-year…

As beliefs are well-known antecedents of teachers' practices, including assessment practices, sound measurement of teacher beliefs is critical for scholarly research as well as practical purposes. The present study examined the validity of inferences derived from the Conceptions of Assessment III--Abridged (COA-IIIA) instrument with US PK-12…