Student evaluations of teaching (SET) have been the principal instrument to elicit students' opinions in higher education institutions. Many decisions, including high-stake ones, are made based on SET scores reported by students. In this respect, reliability of SET scores is of considerable importance. This paper has an argument that there are…

Component Universe Score Profile analysis (CUSP) is introduced in this paper as a psychometric alternative to multivariate profile analysis. The theoretical foundations of CUSP analysis are reviewed, which include multivariate generalizability theory and constrained principal components analysis. Because CUSP is a combination of generalizability…

As a property of test scores, reliability/dependability constitutes an important psychometric consideration, and it underpins the validity of measurement results. A review of interpreter certification performance tests (ICPTs) reveals that (a) although reliability/dependability checking has been recognized as an important concern, its theoretical…

The Stressors about Practicum Inventory, a self-report measure of perceived stressors about the practicum, was designed to provide those responsible for the training of primary school teachers with an informative, inexpensive and psychometrically sound tool. The present study describes the development and validation of the 94-item inventory in a…

Given the rapid growth in the number of students completing the International Baccalaureate (IB) Diploma Programme and the potential for students to be educated entirely through IB programmes, it is important to understand associations between programme participation and cognitive and non-cognitive learning attributes. Through a mixed methods…

The rise of digital game and simulation-based learning applications has led to new approaches in educational measurement that take account of patterns in time, high resolution paths of action, and clusters of virtual performance artifacts. The new approaches, which depart from traditional statistical analyses, include data mining, machine…

This article investigates the quality of social scientific listening research that reports numerical data to substantiate claims appearing in the "International Journal of Listening" between 1987 and 2011. Of the 225 published articles, 100 included one or more studies reporting numerical data. We frame our results in terms of eight…

This article examines the effect of improvements in timing/rhythmicity on mathematics achievement. A total of 86 participants attending 1st through 4th grades completed pre- and posttest measures of mathematics achievement from the Woodcock-Johnson III Tests of Achievement. Students in the experimental group participated in a 4-week intervention…

The Strengths and Difficulties Questionnaire (SDQ) is an instrument developed by Goodman for screening child and adolescent psychopathology. The aim of this study is to contribute to the analysis and validation of the internal structure of the Italian SDQ teacher version (SDQ-T). The SDQ-T was completed by 301 teachers, evaluating 3,302 children…

Classical methods for detecting outliers deal with continuous variables. These methods are not readily applicable to categorical data, such as incorrect/correct scores (0/1) and ordered rating scale scores (e.g., 0,..., 4) typical of multi-item tests and questionnaires. This study proposes two definitions of outlier scores suited for categorical…

Institutions of higher education want to diversify their learning climates, and many offer courses in multiculturalism, yet these courses still do not meet the needs of attitudinal change. A new instrument was developed, the Munroe Multicultural Attitude Scale Questionnaire (MASQUE), that was theoretically based in Banks's transformative approach,…

The purpose of the current study was to investigate the relative performance of the parametric, rank, and normal scores procedures when the classical assumptions were met and under violations of these assumptions. This investigation included the normal scores as well as the rank test. (LMO)

The present study was designed to investigate the discriminant validity of the Devereux Scales of Mental Disorders (DSMD; Naglieri, LeBuffe, & Pfeiffer, 1994) in relation to a widely used behavior rating scale, the Teacher Report Form (TRF; Achenbach, 1991), in children and adolescents with emotional disturbance (ED). A matched sample of 148…

Establishing stability, defined as observing minimal measurement error in a test-retest assessment, is vital to validating psychometric tools. Correlational methods, such as Pearson product-moment, intraclass, and kappa are tests of association or consistency, whereas stability or reproducibility (regarded here as synonymous) assesses the…