In writing assessment, there are quite a number of factors influencing the marking stability and the reliability of the assessment such as the attitude towards marking and consistency of markers, the physical environment, the design of the items, and marking rubrics. Even the methods to train markers have effects on the reliability of the…

The CATO Institute's Policy Analysis "They Spend WHAT? The Real Cost of Public Schools" contends that the figures most commonly associated with spending on K-12 public education do not include all relevant expenditures. It also cites survey evidence suggesting that voters underestimate the cost of education and, when presented with a higher…

The Education Trust research report "Stuck Schools" suggests a framework for identifying chronically low-performing schools in need of turnaround. The study uses Maryland and Indiana to show that some low-performing schools make progress while others remain stagnant. The report has four serious problems of reliability and validity,…

In this technical report, we provide reliability and validity evidence for the easyCBM[R] Reading measures for grade 2 (word and passage reading fluency and multiple choice reading comprehension). Evidence for reliability includes internal consistency and item invariance. Evidence for validity includes concurrent, predictive, and construct…

Primary objective: to develop a scale to measure students' disposition to engage in lifelong learning. Research design, methods and procedures: using items that reflected the components of lifelong learning, we constructed a 14-item scale that was completed by 309 university and vocational college students, who also completed a measure of deep and…

The purposes of this study were to investigate the reliability and construct validity of measures of reading component skills with a sample of adult basic education (ABE) learners, including both native and nonnative English speakers, and to describe the performance of those learners on the measures. Investigation of measures of reading components…

The purpose of this study was two-fold: (1) to provide the evidence for the reliability of the modified Fennema-Sherman Mathematics Attitude Scales (FSMAS), as translated to Turkish language and transformed to the educational technology context, and (2) to investigate high school students' motivation to use technology for learning by a comparative…

A protocol for analyzing the activity structure of a classroom lesson was developed and field tested. The protocol can be completed on site by a person serving as both observer and analyst. Validity of the protocol was established by reference to the large body of qualitative research on activity structure and the analysis of teaching. The…

Two factors which may affect the ratings assigned to an essay test are investigated: (1) context effects; and (2) score level effects. Context effects exist in essay scoring if an essay is rated higher when preceded by poor quality essays than when preceded by high quality essays. A score level effect is defined as a change in the score (value)…

A total of 1,487 eleventh grade students from the Hamburg (West Germany) school system were asked to complete four writing assignments used in an International Association for the Evaluation of Educational Achievement (IEA) study of writing assessment. In analyzing the writing samples, the study focused on: (1) between-rater effects; (2)…

This report gives some simple examples of stability for one factor and 2 x 2 factorial analysis of variance, reliability and correlations. The findings are very different: from superstability (no transformation whatsoever can change the result) to almost total instability. This is followed by a discussion of applications to multivariate analysis,…

Today's psychological measurement depends almost exclusively on the "standardized test." A certain amount of non-standardization, however, exists in the administration of any standardized test, with the amount unknown for any given test score. Time limits on tests pose a bigger problem since another variable is introduced, pressure. Test taking…

Livingston's work is a careful analysis of what occurs when one pools two populations with different means, but similar variances and reliability coefficients. However, his work fails to advance reliability theory for the special case of criterion-referenced testing. See ED 042 802 for Livingston's paper. (MS)

This paper examines the individual consistency of eleventh grade respondents in normal test-retest situations as well as situations involving changed response methods. It postulates the personality-trait "discriminator" (high vs low) and confirms the trait's influence on individual consistency within and across Biology Cognitive…

Observer disagreement is important because it limits the reliabilities of observational measures. To avoid this limitation, observers should be trained, and criterion-related and intraobserver agreement measures should be used both before and during a study. Several agreement coefficients are examined for their applicability. (Author/BW)