The "LFK" index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing the "LFK" index test to three standard tests for funnel plot asymmetry in settings with smaller or larger…

Rett syndrome is a severe neurodevelopmental disorder that affects about 1 in 10,000 females. Clinical trials of disease modifying therapies are on the rise, but there are few psychometrically sound caregiver-reported outcome measures available to assess treatment benefit. We report on a new caregiver-reported outcome measure, the Rett Caregiver…

As diagnostic classification models become more widely used in large-scale operational assessments, we must give consideration to the methods for estimating and reporting reliability. Researchers must explore alternatives to traditional reliability methods that are consistent with the design, scoring, and reporting levels of diagnostic assessment…

The knowledge of the materials to be taught to the students is the basic knowledge that preservice mathematics teachers should possess, as they need to prepare themselves for teaching. In order to research preservice teachers' understanding of the subject matter and teaching skils, valid and reliable test instruments are required. Knowledge of…

The Proposed Specifiers for Conduct Disorder (PSCD) assesses psychopathy components of grandiose-manipulative (GM), callous-unemotional (CU), daring-impulsive (DI), and conduct disorder (CD). Research on PSCD is still in its infancy, and further research is necessary to examine its psychometric properties. We investigated the correlations between…

In order for climate literacy assessments to be applicable in large-scale studies, it is essential that they comply with the standards of test administration while maintaining consistency with a comprehensive definition of the concept. In alignment with the different educational frameworks and the Climate Literacy Principles of the U.S. Global…

Diagnostic classification models (DCMs) have grown in popularity as stakeholders increasingly desire actionable information related to students' skill competencies. Longitudinal DCMs offer a psychometric framework for providing estimates of students' proficiency status transitions over time. For both cross-sectional and longitudinal DCMs, it is…

There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating…

In recent years, the application of explainability techniques to automated essay scoring and automated short-answer grading (ASAG) models, particularly those based on transformer architectures, has gained significant attention. However, the reliability and consistency of these techniques remain underexplored. This study systematically investigates…

The purpose of this study is to identify which scale short-form development method produces better findings in different factor structures. A simulation study was designed based on this purpose. Three different factor structures and three simulation conditions were selected. As the findings of this simulation study, the model-data fit and…

The purpose of this study is to describe and evaluate a multilingual automated essay scoring (AES) system for grading essays in three languages. Two different sentence embedding models were evaluated within the AES system, multilingual BERT (mBERT) and language-agnostic BERT sentence embedding (LaBSE). German, Italian, and Czech essays were…

Teacher design work has gained increasing attention by re-conceptualizing teachers as designers rather than curriculum deliverers. However, assessing teacher design work can be challenging given that there are very few research tools to assess teacher design knowledge (TDK) competencies. To fill that gap, this study proposes a survey that assesses…

Background and Context: As computer science (CS) education expands and the need for well-prepared CS teachers grows, understanding what motivates teachers to teach CS can help address challenges to recruiting, preparing, and retaining teachers. Objective: The goal of this work was to develop and validate a scale that measures teachers' motivation…

This article proposes a conceptual and measurement framework for postpubertal, L2 speech learning aptitude that is centered around domain-general auditory processing (i.e., representing spectral and temporal characteristics of sounds). To this end, we examine the construct and reliability of a battery of auditory processing tests by presenting the…

This paper aims to advocate for a balanced approach to model fit evaluation in structural equation modeling (SEM). The ongoing debate surrounding chi-square test statistics and fit indices has been characterized by ambiguity and controversy. Despite the acknowledged limitations of relying solely on the chi-square test, its careful application can…