This study aimed to measure the effect of rater training given to improve the peer assessment skills of secondary school students on rater behaviors using the many-facet Rasch Measurement model. The research employed a single-group pretest-posttest design. Since all raters scored all students, the analyses were carried out in a fully crossed (s x…

The Canadian Agility and Movement Skill Assessment (CAMSA) is a valid and reliable circuit-based test of motor competence which can be used to assess children's skills in a live or recorded performance and then coded. We aimed to analyze the intrarater reliability of the CAMSA scores (total, time, and skill score) and time measured, by comparing…

Purpose: Multiple-measure principal evaluation systems have become commonplace in the past decade, but we do not know how principals perceive their evaluations under these regimes. This study analyzes how principals perceive evaluation in a state that was an early adopter of such a system. It describes how attitudes are explained by individual and…

Purpose: Multiple-measure principal evaluation systems have become commonplace in the last decade, but we do not know how principals perceive their evaluations under these regimes. This study analyzes how principals perceive evaluation in a state that was an early adopter of such a system. It describes how attitudes are explained by individual and…

In recent decades, the assessment of instructional quality has grown into a popular and well-funded arm of educational research. The present study contributes to this field by exploring first impressions of untrained raters as an innovative approach of assessment. We apply the thin slice procedure to obtain ratings of instructional quality along…

Behavior rating scales represent one of the most commonly used types of assessments in school psychology. Yet, they suffer from a fundamental limitation: They are an indirect methodology influenced partially by student behavior and partially by rater perspectives. Thus, the current study utilized advanced analytic approaches to evaluate rater…

The purpose of this study was to determine the male-female ratio of band directors serving on adjudication panels for Texas University Interscholastic League (UIL) concert and sight reading evaluations from 2010-2019. The Texas Music Forms database was used to access a list of UIL band concert and sight reading adjudicators from across the state…

The study established the validity and reliability of a weighted individual performance-based assessment tool within the utility scope of middle school orchestral strings. The following research questions guided this study: 1. What specific string-playing behaviors and corresponding criteria validate a weighted individual performance-based…

When students present writing tasks that require higher order thinking skills to work, one of the most important problems is scoring these writing tasks objectively. The fact that raters give scores below or above their performance based on several environmental factors affects the consistency of the measurements. Inconsistencies in scoring…

This study investigated the effectiveness of statistical adjustments applied to rater bias in many-facet Rasch analysis. Some changes were first made in the dataset that did not include "rater × examinee" bias to cause to have "rater × examinee" bias. Later, bias adjustment was applied to rater bias included in the data file,…

Automatic machine learning (AutoML) methods automate the time-consuming, feature-engineering process so that researchers produce accurate student models more quickly and easily. In this paper, we compare two AutoML feature engineering methods in the context of the National Assessment of Educational Progress (NAEP) data mining competition. The…

This is the sixth and culminating study in a series whose purpose has been to acquire a conceptual understanding of school band performance and to develop an assessment based on this understanding. With the present study, we cross-validated and applied a rating scale for school band performance. In the cross-validation phase, college students…

This article explores issues that had to be confronted over a three-year period in terms of two New York State funded grants for which I was the evaluator. Major, critical challenges are presented, described, and discussed. Specifically, the paper addresses issues related to lack of planning at the grant design stage that resulted in challenges…

Recent research in collaborative learning has indicated that peer assessment has a positive influence on students' learning performance and learning process. This study investigates how the different subgroups of two roles (assessors and assessees) in peer assessment contribute to students' perceptions of science learning in mobile…

Teacher perceptions after 5 years of implementing evaluation protocols that were initiated under Race to the Top revealed attitudes about the evaluation instrument used and the nature of their relationship with the evaluator. This study surveyed middle and high school teachers in nine Eastern Tennessee school districts. Data indicated unintended…