Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school's…

Using generalizability (G-) theory, this study examined the accuracy and validity of the writing scores assigned to secondary school ESL students in the provincial English examinations in Canada. The major research question that guided this study was: Are there any differences between the accuracy and construct validity of the analytic scores…

Using generalizability theory, this study examined both the rating variability and reliability of English as a second language (ESL) students' writing in two provincial examinations in Canada. This article discusses expected and unexpected similarities and differences related to rating variability and reliability between the two testing programs.…

Prediction effect sizes such as ROC area are important for demonstrating a risk assessment's generalizability and utility. How a study defines recidivism might affect predictive accuracy. Nonrecidivism is problematic when predicting specialized violence (e.g., domestic violence). The present study cross-validates the ability of the Ontario…

The factor structure of a Chinese version of the Zarit Burden Interview was assessed with a random sample of Chinese caregivers in Canada. The results indicated that the latent constructs identified in earlier studies on caregivers did not fit well when tested with Chinese caregivers. Exploratory factor analysis reported a five-factor solution,…

Using generalizability theory, this study examined both the rating variability and reliability of ESL students' writing in the provincial English examinations in Canada. Three years' data were used in order to complete the analyses and examine the stability of the results. The major research question that guided this study was: Are there any…

Possible integrated and independent tasks were pilot tested for the writing section of a new generation of TOEFL® (Test of English as a Foreign Language™) examination. This study examines the impact of various rating designs as well as the impact of the number of tasks and raters on the reliability of writing scores based on integrated and…

The Collis Attitudes toward Computers Survey, developed by B. Collis (1984) for a secondary school population, was tested with 305 college students attending the University of Winnipeg (Canada). Results support the reliability and generalizability of the survey and identified one dominant factor--personal interest and enjoyment of computers. (SLD)

Examined the generalizability of written-response scores on the English 30 diploma examination administered to Alberta 12th-grade students. Student scores differed as a function of rater, but this variance component was small across two tasks and two administrations; score generalizability was high using a two-rater system; and scale variability…

This study examined the generalizability of the internal/external (I/E) frame of reference model of academic self-concept development. The "external" component of the model refers to comparing one's achievement with one's peers; in LISREL causal modeling, this external comparison is presented as positive paths. The "internal"…