The Program of Systematic Evaluation (PSE), which began as an attempt to explore the instructional dimensions of multimedia, has evolved into the most comprehensive set of experimental multimedia studies exploring the varied dimensions of cognitive load theory as they relate to the design of effective and efficient visual learning environments for…

Presents maximum-likelihood confirmatory factor analysis (CFA) as a means for interpreting complete and incomplete multitrait-multimethod designs. CFA was applied to four data sets representing four designs and to an artificial data set to demonstrate its use in test validation designs to assess discriminant, convergent, and construct validity.…

The criteria for assessing convergent and discriminant validity proposed by D. T. Campbell and D. W. Fiske (1959) are shown to be inadequate for either the additive or multiplicative structures of data in a multitrait-multimethod matrix. Model-specific criteria are more promising for assessing convergent and discriminant validity. (Author/SLD)

The evaluation of an intervention delivered across multiple treatment sites presents a unique opportunity for evaluators to gauge the manner and degree to which the "impact" of treatment varies across implementation conditions and different target populations. However, the availability of implementation data for each treatment site, while…

Primary health care nurses are frequently overlooked when delivering services to children with developmental disabilities, despite the fact that they are often the first contact many primary caregivers have with rehabilitation professionals and usually remain the bridge between caregivers and professionals. A time series one group design with…

This paper reports on a commissioned research project to evaluate the impact of support (mainly funding) given by the UK Government's National Endowment for Science, Technology and the Arts (NESTA) to various projects under the general heading of "science learning" over a four-year period (2000-2004). Findings emerging from the study…

In confirmatory factor analysis (CFA), models exist that are admissible and demonstrably identified where the factor loading matrix from latent to manifest variables is of less than full column rank, even though other models are not identified. Implications for CFA are discussed. (SLD)

This study investigates the latent structure of individual differences in vocational, leisure, and family interests. The participants consisted of 302 French adults who rated their like or dislike for activities based on Holland's RIASEC typology and presented in three life domains: work, family and leisure. The multitrait-multimethod model…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…

The Life-Patterns project is a panel-cohort longitudinal study following the life trajectories of 1,908 young people who left school in Victoria in 1991. The project attempts to obtain nuanced pictures of young people's lives in three ways: using a reflexive longitudinal study design, with opportunities for participant feedback and influence on…

Leadership depends on the situation. Few social scientists would dispute the validity of this statement. But the statement can be interpreted in many different ways, depending, at least in part, on what one means by leadership. This article begins with a definition of leadership and a brief description of 3 historically important theories of…

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT). An item-level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003-2004 TOEFL iBT Field…

During the last 15 years, there has been a steady increase in the popularity and sophistication of the confirmatory factor analysis approach to multitrait-multimethod (MTMM) data. However, important problems exist, the most serious being the ill-defined solutions that plague MTMM studies and the assumption that so-called method factors primarily…

Multiple evaluation methods (MEMs) can cause an inferential challenge, although there are strategies to strengthen inferences. Practical and theoretical issues involved in the use by social scientists of MEMs, three potential problems in drawing inferences from MEMs, and short- and long-term strategies for alleviating these problems are outlined.…

An equal-level approach is proposed for investigating multitrait-multimethod (MTMM) matrices with respect to other organizational units that contain additional information concerning a MTMM matrix's validity. The approach requires equality in "data level" before coefficients are submitted for evaluation. Disaggregation is central to…