Studied the stability of accountability scores and gains for schools using fourth-grade reading results for schools (samples from 757 to 802 schools) in Colorado for 4 years of administration of the Colorado Student Assessment Program. Year-to-year changes were found to be quite unstable, resulting in a near zero correlation of the school gains…

Compared approaches to determining the precision of gain scores: (1) the asymptotic normal distribution of the maximum likelihood estimator of the person parameter; and (2) the exact conditional distribution of the gain score. Use of three data sets illustrates that these methods yield more relevant and more detailed information than traditional…

Analyzes the substitution process in the labor market to develop an educational classification based on observed possibilities of workers with different educational backgrounds to enter similar occupations using the recognizability of the groups distinguished and incorporating the criterion of statistical reliability. Results in an educational…

As an alternative to equally weighted difference scores, examined an orthogonal reliable component analysis (RCA) solution and an oblique principal components analysis (PCA) solution for the standardization sample of the Kaufman Assessment Battery for Children (KABC; A. Kaufman and N. Kaufman, 1983). Discusses the practical implications of the…

Tests consistency of self-reports of alcohol and drug use by comparing responses incarcerated youth give to corrections staff and to university research interviewers only weeks apart. Significantly more incarcerated adolescents (n = 894) report lifetime drug and alcohol use to researchers than to correctional staff. Sex and ethnic differences were…

Discusses general issues and concerns relative to the adaptation of paper-pencil assessment instruments to computerized formats. Describes and evaluates Self-Directed Search computerized version (SDS-CV). Presents strengths and weaknesses of the SDS-CV and makes recommendations for its use. (Author/ABL)

Job analysts observed and interviewed job incumbents representing 20 diverse occupations and rated each occupation on a wide variety of characteristics to determine the reliability of current job analysis methods and to compare these methods with a new scale. Scales representing broad, abstract job characteristics tended to have higher…

The potential for bias in reliability estimation and for errors in item selection when alpha or unit-weighted omega coefficients are used is explored under simulated conditions. Results suggest that composite reliability may be an assessment tool but should not be an item selection tool in structural equations. (SLD)

Evaluates four writing samples from each student in several freshman composition courses. Finds that students did not perform consistently from one assignment to the next. Suggests that knowing how well a student performs on one writing task says very little about how well the student is likely to perform on other writing tasks. (RS)

Today's employers desire a number of qualities in their workers: self-confidence, technical knowledge, positive attitude, communication skills, teamwork, and the ability to learn. Also important are the various aspects of reliability: punctuality, attendance, commitment to excellence, and listening skills. (SK)

Discusses the role of feedback in performance technology and suggests ways to improve both the accuracy and the effectiveness of the feedback. Desirable characteristics of feedback are described, including promptness, directness, reliability, and usability; and the need for assessment and assimilation following feedback is explained. (LRW)

Addresses two vital issues in international studies of educational achievement: the quality of data obtained from participating countries and the interpretation of those data. Contends that examples of problems in item development for international studies are related to the quality of the data and norms used in interpretation. (Author/NB)

Addresses concerns about the trustworthiness of research results derived from image-based data, particularly questions of validity assessment. Compares examples of such research to provide an illustration of these issues. Presents a tentative strategy for ensuring trustworthiness based on five criteria: completeness, interpretive adequacy,…

The Tower of London test (T. Shallice, 1982), a measure of executive function, was reconstructed to increase its reliability through revisions tested with successive samples of 50, 50, and 34 college students. Adjusting the item pool resulted in acceptable test-retest reliability. (SLD)

Evaluated external assessor ratings of the quality of the research team and the proposed research for proposals submitted to the Australian Research Council large grants program. Data from 313 evaluations show that the reliability of the assessments varied systematically with the number of assessors and the score considered. Discusses implication…