In signal detection rater models for constructed response (CR) scoring, it is assumed that raters discriminate equally well between different latent classes defined by the scoring rubric. An extended model that relaxes this assumption is introduced; the model recognizes that a rater may not discriminate equally well between some of the scoring…

Businesses and other organizations across the globe are becoming more and more data-driven, using a combination of descriptive, diagnostic, predictive and prescriptive analytics to gain a strategic advantage through understanding the past, what we hope to happen in the future, and the ability to accurately predict future outcomes. These forms of…

Despite the call for an argument-based approach to validity over 25 years ago, few examples exist in the published literature. One possible explanation for this outcome is that the complexity of the argument-based approach makes implementation difficult. To counter this claim, we propose that the Assessment Triangle can serve as the overarching…

Designing a theory-based assessment with sound psychometric qualities to measure a higher-order cognitive construct is a highly desired yet challenging task for many practitioners. This paper proposes a framework for designing a theory-based assessment to measure a higher-order cognitive construct. This framework results in a modularized yet…

When the coronavirus pandemic was declared a public health crisis in March 2020, school psychologists were forced into situations where face-to-face interaction with their students was discouraged and in some cases, prohibited. Consequently, the traditional practice of school psychology abruptly ended. Individualized Education Plans (IEP) and…

As teachers are increasingly asked to respond to assessment and evaluation demands in their schools, the aim of this article is to provide suggestions and scales for singing accuracy. A description of each step in the assessment design process is provided, along with specific suggestions for a singing development scale and a singing accuracy scale.

This article aims to provide a theoretical framework for computerized adaptive tests (CAT) and item response theory models for polytomous items. Besides that, it aims to introduce the simulation and live CAT software to the related researchers. Computerized adaptive test algorithm, assumptions of item response theory models, nominal response…

The Task Model is a description of each task in a workflow. It defines attributes associated with that task. The creation of task models becomes increasingly important as the assessment tasks become more complex. Explicitly delineating the impact of task variables on the ability to collect evidence and make inferences demands thoughtfulness from…

With the help of teachers and leaders throughout the state, the Indiana Department of Education has developed an optional model teacher evaluation system named RISE. Whether corporations choose to adopt RISE or a model of their own, the department's goal is to assist corporations in developing or adopting models that both comply with IC 20-28-11.5…

The release of institution-level earnings information as part of the Obama Administration's new College Scorecard data has already spawned new "value-added" rankings based on the economic outcomes of students who attended similar institutions. These emerging rankings are an improvement on simple unadjusted rankings, but the wide variance…

The subject of the research is to build methodologies to evaluate the student knowledge by testing. The author points to the importance of feedback about the mastering level in the learning process. Testing is considered as a tool. The object of the study is to create the test system models for defence practice problems. Special attention is paid…

Cultural consensus theory (CCT) is a data aggregation technique with many applications in the social and behavioral sciences. We describe the intuition and theory behind a set of CCT models for continuous type data using maximum likelihood inference methodology. We describe how bias parameters can be incorporated into these models. We introduce…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Over the last several decades, researchers have studied many and varied aspects of rater cognition. Those interested in pursuing basic research have focused on gaining an understanding of raters' thought processes as they score different types of performances and products, striving to understand how raters' mental representations and the cognitive…

This brief technical note describes how to construct item vector plots for dichotomously scored items fitting the multidimensional three-parameter logistic model (M3PLM). As multidimensional item response theory (MIRT) shows promise of being a very useful framework in the test development life cycle, graphical tools that facilitate understanding…