To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

In the last twenty-five years, the discussion surrounding validity evidence has shifted both in language and scope, from the work of Messick and Kane to the updated Standards for Educational and Psychological Testing. However, these discussions haven't necessarily focused on best practices for different types of instruments or assessments, taking…

The goal of this paper is twofold. First, the paper clarifies and elaborates on an important theoretical construct called orientation with respect to understanding in mathematics, which denotes the degree to which students exhibit an inclination towards and demonstrate an earnest concern for understanding in mathematical learning. Second, the…

Dynamic Measurement Modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate students' capacity to learn over the course of an educational program (i.e., growth scores). Here, we provide justification for this approach in health professions education and demonstrate its proof of concept with three time-points of USMLE Step…

This study demonstrates the importance of Rasch Measurement Theory (RMT) in program evaluation when outcome measures need to be constructed from scratch. The paper introduces typical measure validation methods presented in program evaluation texts and discusses room for improvement. The study then illustrates how the seamless transitions from…

The main purpose of the study is to evaluate the qualities of human essay ratings for a large-scale assessment using Rasch measurement theory. Specifically, Many-Facet Rasch Measurement (MFRM) was utilized to examine the rating scale category structure and provide important information about interpretations of ratings in the large-scale…

Existing studies have defined and assessed disciplinary literacy, mathematical literacy, and general academic language. However, there is a need to define and assess mathematics discipline-specific language (MDL), particularly for elementary school teachers. Therefore, the purpose of this study was to develop a research instrument to assess the…

This study, Creative Behavior Observation Form was developed to assess creativity of the children. While the study group on the reliability and validity of Creative Behavior Observation Form was being developed, 257 children in total who were at the ages of 5-6 were used as samples with stratified sampling method. Content Validity Index (CVI) and…

One commonly used measure of lexical sophistication is the Advanced Guiraud (AG; [9]), whose formula requires frequency band counts (e.g., COCA; [13]). However, the accuracy of this measure is affected by the particular 2000-word frequency list selected as the basis for its calculations [27]. For example, possible issues arise when frequency lists…

Interactions between teachers and students contribute to the quality of the classroom setting and growing evidence that the affective undertone of these interactions matters when considering outcomes for both the students and the teachers they serve. Although numerous measures of teacher-student interactions exist, relying on self-report or…

This study extends research on graduate student development by examining descriptive findings and validity of a self-report survey designed to capture graduate students' assessments of their teaching and research skills. Descriptive findings provide some information about areas of growth among graduate students' in the first years of their…

After decades of disinterest, evaluation of the performance of elementary and secondary teachers in the United States has become an important educational policy issue. As U.S. states and districts have tried to upgrade their evaluation processes, one of the models that has been increasingly used is the Framework for Teaching. This paper summarizes…

A number of authors have presented data that challenge the validity of self-report of strategy use or choice of strategy. We created a multiple-choice measure of students' strategy use based on the work of Kozminsky, E., and Kozminsky, L. (2001), and tested it with three samples as part of a series of studies testing the fit of the DIME model of…

The purpose of the current investigation was to establish the validity of microanalytic measures used to assess students' self-regulation of an academic science task, not only in terms of immediate achievement, but also in terms of a well-established "person" measure of self-regulated learning. Person measures are designed to capture enduring…

Through a synthesis of test publisher norms and national longitudinal datasets, this study provides new national norms of academic growth in K-12 reading and math that can be used to reinterpret conventional effect sizes in time units. We propose d' a time-indexed effect size metric to estimate how long it would take for an "untreated"…