Purpose: The primary objective of this research paper was to examine the objectivity of the preselected items evaluation (PIE) method, a prevalent translation scoring method deployed by international institutions such as UAntwerpen, UGent and the University of Granada. Design/methodology/approach: This research critically analyzed the scientific…

In this perspective, the authors critically examine "rater training" as it has been conceptualized and used in medical education. By "rater training," they mean the educational events intended to "improve" rater performance and contributions during assessment events. Historically, rater training programs have focused…

Measurement invariance (MI) is an essential part of validity evidence concerned with ensuring that tests function similarly across groups, contexts, and time. Most evaluations of MI involve multigroup confirmatory factor analyses (MGCFA) that assume simple structure. However, recent research has shown that constraining non-target indicators to…

Since 2018, institutions of higher education have been aware of the "enrollment cliff" which refers to expected declines in future enrollment. This paper attempts to describe how prepared institutions in Ohio are for this future by looking at trends leading up to the anticipated decline. Using IPEDS data from 2012-2022, we analyze trends…

As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

Appetite for entrepreneurship education (EE) among vocational students has surged dramatically, driven by persistent challenges of unemployment. As a result, vocational institutions are increasingly focused on how much entrepreneurship exposure students receive, particularly how frequently instructors impart core business knowledge and skills to…

Considerable literature is devoted to teachers' assessment use to support teaching and learning. The study examined the factor structure of a measure of teachers' assessment use along the assessments "of", "for", and "as" learning purpose dimensions. The study also examined the factor structure of teachers' perceived…

In cognitive neurosciences, fundamental principles of mental processes and functional brain organization have been established with highly controlled tasks and testing environments. Recent technical advances allowed the investigation of these functions and their brain mechanisms in naturalistic settings. The diversity in those approaches have been…

We review common situations in Bayesian latent variable models where the prior distribution that a researcher specifies differs from the prior distribution used during estimation. These situations can arise from the positive definite requirement on correlation matrices, from sign indeterminacy of factor loadings, and from order constraints on…

The assessment of human performance is not a new phenomenon. We have evidence that people have been required to prove their worth dating back at least to the Epic of Gilgamesh. What has changed, at least on a large scale, is the importance given to quantitative evidence in the evaluation process. For example, many employers have begun subjecting…

The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting-edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely,…

Given the rapid adoption of machine learning methods by education researchers, and the growing acknowledgment of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing on an integrative literature review and extending a…

This article, located in the discipline of academic literacy studies, draws upon the fields of critical realism, design research, and evaluation studies. It reports on the validation of a flexible evaluation design for assessing the impact of academic literacy interventions. The design was validated in two ways. Firstly, through a process of…

Validity of performance standards is a key element for the defensibility of standard setting results, and validating performance standards requires collecting multiple pieces of evidence at every step during the standard setting process. This study employs a statistical procedure, latent class analysis, to set performance standards and compares…

Surveys are typical for student evaluation of teaching (SET). Survey research consistently confirms the negative impacts of careless responses on research validity, including low data quality and invalid research inferences. SET literature seldom addresses if careless responses are present and how to improve. To improve evaluation practices and…