The purpose of this research is to first estimate the item and ability parameters and the standard error values related to those parameters obtained from Unidimensional Item Response Theory (UIRT), bifactor (BIF) and Testlet Response Theory models (TRT) in the tests including testlets, when the number of testlets, number of independent items, and…

This paper presents the findings of a study designed to explore qualification users' perceptions and experiences of reliability in the context of national assessment outcomes in England. The study consisted of 17 focus groups conducted across six sectors of qualification users: students, teachers, trainee teachers, job-seekers, employers and…

Evidence from recent research suggests that in the UK the public perception of errors in national examinations is that they are simply mistakes; events that are preventable. This perception predominates over the more sophisticated technical view that errors arise from many sources and create an inevitable variability in assessment outcomes. The…

A total of 19 graduate students enrolled in a graduate course conducted 6 consecutive administrations of the Wechsler Intelligence Scale for Children, 4th edition (WISC-IV, Canadian version). Test protocols were examined to obtain data describing the frequency of examiner errors, including administration and scoring errors. Results identified 511…

Critics of standardized testing have recommended replacing standardized tests with more authentic assessment measures, such as classroom assignments, projects, or portfolios rated by a panel of raters using common rubrics. Little research has examined the consistency of scores across multiple authentic assignments or the implications of this…

The usual formulas for the reliability of differences between two test scores are based on the assumption that the error scores are uncorrelated. Formulas are presented for the general case where this assumption is unnecessary. (Author/JKS)

A model which gives maximum likelihood estimates of measurement error within the context of a simplex model for practice effects is presented. The appropriateness of the model is tested for five traits, and error estimates are compared to the classical formula estimates. (Author/JKS)

Lord's, Millman's and Saupe's methods of approximating the standard error of measurement are reviewed. Through an empirical demonstration involving 200 university classroom tests, all three approximations are shown to be biased. (Author/JKS)

Described and listed herein with concomitant sample input and output is the Fortran IV program which estimates parameters and standard errors of estimate per parameters for parameters estimated through multiple matrix sampling. The specific program is an improved and expanded version of an earlier version. (Author/BJG)

A procedure is presented for the analysis of rating data with correlated intrajudge and uncorrelated interjudge measurement errors. Correlations between true scores on different rating dimensions, reliabilities for each judge on each dimension and correlations between intrajudge errors can be estimated given a minimum of three raters and two…