In this digital ITEMS module, Dr. Sue Lottridge, Amy Burkhardt, and Dr. Michelle Boyer provide an overview of automated scoring. Automated scoring is the use of computer algorithms to score unconstrained open-ended test items by mimicking human scoring. The use of automated scoring is increasing in educational assessment programs because it allows…

There are relatively few examples of quantitative approaches to quality control in educational assessment and accountability contexts. Among the several techniques that are used in other fields, Shewart charts have been found in a few instances to be applicable in educational settings. This paper describes Shewart charts and gives examples of how…

Evidence from recent research suggests that in the UK the public perception of errors in national examinations is that they are simply mistakes; events that are preventable. This perception predominates over the more sophisticated technical view that errors arise from many sources and create an inevitable variability in assessment outcomes. The…

If a person were to ask an educator to identify the two most important attributes of an education test, the response most certainly would be "validity and reliability." These two tightly wedded concepts have become icons in the field of education assessment. As far as validity is concerned, the term doesn't refer to the accuracy of a test. Rather,…

This publication presents information on tertiary education and training during 2010, including statistics on participation and outcomes. The definition of tertiary education and training adopted for this publication is formal study in vocational education and training (VET) and higher education, including enrolments in Australian Qualifications…

"Scaling" is the process of constructing a score scale that associates numbers or other ordered indicators with the performance of examinees. Scaling typically is conducted to aid users in interpreting test results. This module describes different types of raw scores and scale scores, illustrates how to incorporate various sources of…

Student assessment is central to the educational process and can be used for multiple purposes including, to promote student learning, to grade student performance and to evaluate the educational quality of qualifications. It is, therefore, of utmost importance that assessment instruments are of a high quality. In this article, we present various…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Preffi 2.0 is an evidence-based Dutch quality assessment instrument for health promotion interventions. It is mainly intended for both planning and assessing one's own projects but can also be used to assess other people's projects (external use). This article reports a study on the reliability of Preffi as an external quality assessment…

The purpose of this paper is to provide an overview of the analysis of data collected by the National Assessment of Educational Progress (NAEP). In simplest terms, the analysis can be characterized as establishing baseline estimates of the percentages of young Americans possessing certain skills, knowledge, understandings, and attitudes and…

A scoring rubric transition study is described. It was designed to evaluate possible drift in scoring the Maryland Writing Test from year to year (when using a modified holistic scoring method), to evaluate strategies for revising swing rubrics from narrative and explanatory writing while maintaining original scoring standards, and to establish…

Unobtrusive measures are recommended as a means of assessing educational outcomes of colleges. Such measures can counteract the response bias which is common in questionnaires and interviews. Outcomes researchers are, in fact, asked to supplement standard measures with unobtrusive measures. Interesting data may result from observation of students'…

This paper provides an overview of common sampling methods (both the good and the bad) likely to be used in community college self-evaluations and presents the results from several simulated trials. The report begins by reviewing various survey techniques, discussing the negative and positive aspects of each method. The increased accuracy and…