The use of the aggregate scoring method for scoring concordance tests requires the weighting of test items to be derived from the performance of a group of experts who take the test under the same conditions as the examinees. However, the average score of experts constituting the reference panel remains a critical issue in the use of these tests.…

Post-secondary data literacy education is critical to students' academic and career success. However, the literature has not adequately addressed the conceptualization and assessment of data literacy for post-secondary students. In this study, we introduced a novel digital performance-based assessment for teaching and evaluating post-secondary…

Digitally-based assessments create opportunities for collecting moment to moment information about how students are responding to assessment items. This information, called log or process data, has long been regarded as a vast and valuable source of data about student performance. Despite repeated assurances of its vastness and value, process data…

Child observation is a critical component of quality pedagogy in early childhood education and care (ECEC). The ORICL (Observe, Reflect, Improve Children's Learning) tool was co-designed by ECEC researchers, policymakers, leaders, and practitioners to support this work. Educators rate the experiences of individual children, and responses of…

The distinguishing feature of dynamic assessment (DA) is the dialectical integration of assessment and instruction. However, how to design the targeted instruction or mediation has been relatively underexplored. To address this gap, this study proposes the attribute-based mediation model (AMM), an English-as-a-foreign-language listening mediation…

Maintaining equivalent performance standards across forms is a psychometric challenge exacerbated by small samples. In this study, the accuracy of two equating methods (Rasch anchored calibration and nominal weights mean) and four anchor item selection methods were investigated in the context of very small samples (N = 10). Overall, nominal…

Student Experience surveys are used to measure student attitudes towards their campus as well as to initiate conversations for institutional change. Validity evidence to support the interpretations of these surveys' results, however, is lacking. The first purpose of this study was to compare three Differential Item Functioning (DIF) methods on…

Purpose: The Access to Literacy Assessment System--Phonological Awareness (ATLAS-PA) was developed for use with children with speech and/or language impairment. The subtests (Rhyming, Blending, and Segmenting) are appropriate for children who are 3-7 years of age. ATLAS-PA is composed entirely of receptive items, incorporates individualized levels…

Purpose: The Access to Literacy Assessment System--Phonological Awareness (ATLAS-PA) was developed for use with children with speech and/or language impairment. The subtests (rhyming, blending, segmenting) are appropriate for children who are 3 to 7 years of age. ATLAS-PA is comprised entirely of receptive items, incorporates individualized levels…

The framework and tools used for classroom assessment can have significant impacts on teacher practices and student achievement. Getting assessment right is an important component in creating positive learning experiences and academic success. Recent government reports (e.g., United States, Australia) call for the development of systems that use…

This study defines subpopulation item parameter drift (SIPD) as a change in item parameters over time that is dependent on subpopulations of examinees, and hypothesizes that the presence of SIPD in anchor items is associated with bias and/or lack of invariance in three psychometric outcomes. Results show that SIPD in anchor items is associated…

This manuscript chronicles the process and products of a redesign for evaluation of the graduate college experience (GCE) which was initiated by a university graduate college, based on its observed need to reconsider and update its measures and methods for assessing graduate students' experiences. We examined the existing instrumentation and…

Analyzing examinees' responses using cognitive diagnostic models (CDMs) has the advantage of providing diagnostic information. To ensure the validity of the results from these models, differential item functioning (DIF) in CDMs needs to be investigated. In this article, the Wald test is proposed to examine DIF in the context of CDMs. This study…

The present study aimed to represent the innovative assessments that support students' learning in STEM education through using the integrative framework for Cognitive Diagnostic Modeling (CDM). This framework is based on three components, cognition, observation, and interpretation (National Research Council, 2001). Specifically, this dissertation…

Valid and reliable standardized assessment of nursing competencies is needed to monitor the quality of vocational education and training (VET) in nursing and evaluate learning outcomes for care work trainees with increasingly heterogeneous learning backgrounds. To date, however, the modeling of professional competencies has not yet evolved into…