Although traditional methods to collect similarity data (for multidimensional scaling [MDS]) are robust, they share a key shortcoming. Specifically, the possible pairwise comparisons in any set of objects grow rapidly as a function of set size. This leads to lengthy experimental protocols, or procedures that involve scaling stimulus subsets. We…

This study explored motivation and engagement among North American (the United States and Canada; n = 1,540), U.K. (n = 1,558), Australian (n = 2,283), and Chinese (n = 3,753) secondary school students. Motivation and engagement were assessed via students' responses to the Motivation and Engagement Scale-High School (MES-HS). Confirmatory factor…

A simulation study compared the performance of robust normal theory maximum likelihood (ML) and robust categorical least squares (cat-LS) methodology for estimating confirmatory factor analysis models with ordinal variables. Data were generated from 2 models with 2-7 categories, 4 sample sizes, 2 latent distributions, and 5 patterns of category…

The process of constructing assessment scales for performance testing is complex and multi-dimensional. As a result, a number of different approaches, both empirically and intuitively based, are open to developers. In this paper we outline the approach taken in the revision of a set of assessment scales used with speaking tests, and present the…

The purpose of this study is to examine the interactions among measurement theories, writing theories, and writing assessments in the United States from an historical perspective. The assessment of writing provides a useful framework for examining how theories influence, and in some cases fail to influence actual practice. Two research traditions…

This paper develops an empirical methodology for the construction of a synthetic multi-dimensional cross-country comparison of the performance of governments around the world in improving the livelihood of their younger population. The devised "Youth Welfare Index" is based on the nonparametric Data Envelopment Analysis (DEA) methodology and…

Multidimensional scaling (MDS) is a versatile technique for understanding the structure of multivariate data. Recent studies have applied MDS to the problem of evaluating content validity. This paper describes the importance of evaluating test content and the logic of using MDS to analyze data gathered from subject matter experts employed in…

This paper begins by reviewing some of the different models of third-party observation of university teaching that can be found in the literature. Having analysed these, it argues that--if "peer" is taken to indicate equality of status--only one is genuinely a model of peer-observation. It proposes an alternative categorisation of third-party…

A method of optimal scaling for multivariate ordinal data--OSMOD--is described, within a generalized principal component analysis. It yields a: multidimensional configuration of items, unidimensional scale of category weights for each item, and multidimensional configuration of subjects. OSMOD involves solving an eigenvalue problem and executing a…

As a result of the "No Child Left Behind Act," there is increased emphasis on assessment in P-12 schools today. School administrators are responsible for assessing student achievement, teacher and staff effectiveness, school climate, and graduates' perceptions of school programs. At the university level, organizations, such as the National Council…

Describes content-validity theory and illustrates new and traditional approaches for conducting content-validity studies. Newer approaches are based on multidimensional scaling analysis of item-similarity ratings, while traditional approaches are based on ratings of item-objective congruence and relevance. (Author/SLD)

A study was designed to test ordinal and interval properties of rating scales, similar to those commonly employed in evaluation research. The analysis was based on data generated by the use of paired comparisons and direct scales. The conclusion was reached that such rating scales appear to possess metric properties. (Author/LMO)

Attitude measurement has had a somewhat chequered history since the possibility of achieving it successfully was demonstrated by Thurstone in 1929. It has been an important area in science education, particularly in the context of falling uptakes in the physical sciences in many countries, and there have been many attempts to measure learner…

Illustrated is the power of multidimensional scaling in reducing a complex set of proximity measures to a simple geometric picture that shows the relationship among data objects. Methods are discussed for determining the number of dimensions needed to represent a set. (Author/DEP)

This paper discusses problems associated with the application of experimental methods to program evaluation, proposes an alternative approach (program-attribute scaling), and reports the procedure and results of a trial application of the alternative approach. (Author)