Reading-to-write (RTW) tasks are becoming increasingly popular and have already been used in several high-stakes English proficiency exams, either replacing or complementing a prompt-based essay test. However, it is still not clear that what accounts for successful or unsuccessful performance on an integrated reading-writing task is owing to the…

Purpose: The purpose of this study was to examine the reliability of, and sources of variability in, language measures from interviews collected from young school-age children. Method: Two 10-min interviews were collected from 20 at-risk kindergarten children by an examiner using a standardized set of questions. Test-retest reliability…

Written by a social worker for social work students, this is a nuts and bolts guide to statistics that presents complex calculations and concepts in clear, easy-to-understand language. It includes numerous examples, data sets, and issues that students will encounter in social work practice. The first section introduces basic concepts and terms to…

Multivariate methods are being used with increasing frequency in educational research because these methods control "experimentwise" error rate inflation, and because the methods best honor the nature of the reality to which the researcher wishes to generalize. This paper: explains the basic logic of canonical analysis; illustrates that…

The present paper suggests that multivariate techniques are very important in social science research, and that canonical correlation analysis may be particularly useful. The logic of canonical analysis is explained and discussed. The necessity of using replicability/generalizability analyses is argued. It is suggested that cross-validation…

The main purpose of this study is to examine the content structure of the Multistate Bar Examination (MBE) using the "table of specifications" model from the perspective of multivariate generalizability theory. Specifically, using MBE data collected over different years (six administrations: three from the February test and three from July test),…

Statistical invariance procedures provide a way of looking at the generalizability of research results from sample to sample when the research has not been validated by replication. This paper discusses the Procrustean Rotation invariance procedure following a canonical correlation analysis. The computer program RELATE is used to gauge the…

The need for using invariance procedures to establish the external validity or generalizability of statistical results has been well documented. Invariance analysis is a tool that can be used to establish confidence in the replicability of research findings. Several approaches to invariance analysis are available that are broadly applicable across…

Although multivariate generalizability theory was developed more than 30 years ago, little published research utilizing this framework exists and most of what does exist examines tests built from tables of specifications. In this context, it is assumed that the universe scores from levels of the fixed multivariate facet will be correlated, but the…

This report describes 12 studies dealing with the knowledge monitoring component of metacognition. It is assumed that knowledge monitoring is basic to other metacognitive activities, such as evaluating learning, selecting appropriate strategies, or planning, because distinguishing between what students know and do not know ought to be a…