This study examines the use of cross-classified random effects models (CCrem) and cross-classified multiple membership random effects models (CCMMrem) to model rater bias and estimate teacher effectiveness. Effect estimates are compared using CTT versus item response theory (IRT) scaling methods and three models (i.e., conventional multilevel…

The majority of large-scale assessments develop various score scales that are either linear or nonlinear transformations of raw scores for better interpretations and uses of assessment results. The current formula for coefficient alpha (a; the commonly used reliability coefficient) only provides internal consistency reliability estimates of raw…

Psychometric practice does not always converge with the advances of psychometric theory. In order to investigate this gap, the authors focus on the 10 most used psychological tests in Europe, as identified by recent surveys. The article analyzes test manuals published in 6 different European countries for these 10 most used tests. A total of 32…

Four types of metric scales are distinguished: absolute, ratio, difference, and interval. A general coefficient of association for two variables of the same scale type is developed which reduces to specific coefficients of association for each scale type. (NSF)

The axioms of a three-set Guttman simplex model are presented and the effects of relaxing the axioms for one of the three sets are examined. This model can be used to define longitudinal developmental scales. (NSF)

Reliability maximizing weights are related to theoretically specified true score scaling weights to show a constant relationship that is invariant under separate linear tranformations on each variable in the system. Test theoretic relations should be derived for the most general model available and not for unnecessarily constrained models.…

Various models have been proposed for analyzing dichotomous test or questionnaire items which were constructed to reflect an assumed underlying structure (e.g., hierarchical). This paper shows that many such models are special cases of latent class analysis and discusses a currently available computer program to analyze them. (Author/JKS)

Considers the relationship between coordinate estimates in components analysis and multidimensional scaling. Reports three small Monte Carlo studies comparing nonmetric scaling solutions to components analysis. Results are related to other methodological issues surrounding research on the general ability factor, response tendencies in…

Two methods of constucting equal-interval scales for educational achievement are discussed: Thurstone's absolute scaling method and Item Response Theory. Alternative criteria for choosing a scale are contrasted. It is argued that clearer criteria are needed for judging the appropriateness and usefulness of alternative scaling procedures.…

Score equating requires that the forms to be equated are functionally parallel. That is, the two test forms should rank order examinees in a similar fashion. In language proficiency testing situations, this assumption is often put into doubt because of the numerous tests that have been proposed as measures of language proficiency and the…

Estimation techniques for generating the covariance matrix for two new tests and an existing test without the necessity of any examinee having to take two complete tests is presented. An application of these techniques to linear, observed-score, test equating is presented. (Author/JKS)

An attempt was made to determine which item response theory (IRT) equating method results in the least amount of equating error or "scale drift" when equating scores across one or more test forms. An internal anchor test design was employed with five different test forms, each consisting of 30 items, 10 in common with the base test and 5…

The nonparametric approach to test theory discussed here has its roots in the early work of Guttman, Lazarsfeld, and Meredith; and more recently in the work of Cliff and in Tatsuoka and Tatsuoka. Mokken's extensive treatment of this subject concentrated on defining, constructing, and testing unidimensional scales, based on responses to dichotomous…

A sample of 234 fifth- and 259 sixth-grade students scaled the items of the Piers-Harris, Tennessee, Coopersmith, and Lipsett self-concept measures. The scaling of the Piers-Harris and the Tennessee inventories was examined in reference to their subscales. The present technique placed items on a bivariate plane of two orthogonal dimensions…

An important principle in constructing rating scales is to develop items that reflect various degrees of the "pro" (positive) and "contra" (negative) aspects of the trait being measured. Where both positive and negative items are pooled, they can be arranged in order along the trait continuum, but for classical and item…