This study aimed to measure the effect of rater training given to improve the peer assessment skills of secondary school students on rater behaviors using the many-facet Rasch Measurement model. The research employed a single-group pretest-posttest design. Since all raters scored all students, the analyses were carried out in a fully crossed (s x…

When students present writing tasks that require higher order thinking skills to work, one of the most important problems is scoring these writing tasks objectively. The fact that raters give scores below or above their performance based on several environmental factors affects the consistency of the measurements. Inconsistencies in scoring…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

Teacher observations have become a national education phenomenon, mandated by federal policies and promoted by philanthropists. They are crucial components of teacher evaluation systems that often have high stakes for teachers and school systems, but have sparked little innovation. Recent calls to make teacher evaluations better, faster, and…

Researchers have expressed concern about "implementation" fidelity in intervention research but have not extended that concern to "assessment" fidelity, or the extent to which pre-/posttests are administered and interpreted as intended. When studying reading interventions, data gathering heavily influences the identification of…

We evaluated the reliability and validity of two oral reading fluency scores for 1-minute equated passages: median score and mean score. These scores were calculated from measures of reading fluency administered up to five times over the school year to students in grades six to eight (n = 1,317). Both scores were highly reliable with strong…

Background: Professional standards for validity of achievement tests have long reflected a consensus that validity is the degree to which evidence and theory support interpretations of test scores entailed by the intended uses of tests. Yet there are convincing lines of evidence that the standards are not adequately followed in practice, that…

Scientifically based research used to inform evidence based school reform efforts has been required by the federal government in order to receive grant funding since the reenactment of No Child Left Behind (2002). Educational evaluators are thus faced with the challenge to use rigorous research designs to establish causal relationships. However,…

In this study, the purpose was to determine whether there was concordance among raters in the assessment of the writing performance using analytic rubric; furthermore, factors affecting the assessment process were examined. The analytic rubric used in the study consists of three sections and ten properties: External structure (format, spelling and…

This is the first report on the evaluation of the Inquiry Based Science and Technology Education Program (IN-STEP), an innovative and ambitious science education initiative for lower secondary schools being undertaken by a public-private partnership in Thailand funded by MSD-Thailand, an affiliate of Merck & Co. IN-STEP is a public-private…