This study investigates technical efficiency and productivity for Swedish higher education institutions (HEIs). One identified problem in previous research concerns adjusting efficiency scores for input quality. This problem is avoided using grades from upper-secondary schools. A second problem concerns heterogeneity with respect to subjects and…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

Frustrations with traditional testing led a group of teachers at the English for adults program at Universidad EAFIT (Colombia) to design tests aligned with the institutional teaching philosophy and classroom practices. This article reports on a study of an item-by-item evaluation of a series of English exams for validity and reliability in an…

Purpose: The purpose of this paper is to assess the research efficiency of the Chilean higher education institutions (HEIs). As it has been argued in the literature, universities in Chile are far from being considered research-oriented institutions. Current governmental reforms have put pressures on the efficient use of public resources,…

We explain how to simulate both univariate and bivariate raw data sets having specified values for common summary statistics. The first example illustrates how to "construct" a data set having prescribed values for the mean and the standard deviation--for a one-sample t test with a specified outcome. The second shows how to create a bivariate data…

A procedure for estimating the degree to which a subtest uniquely contributes to total test performance is presented and discussed. Uniqueness analysis may be appropriately applied to any composite measurement instrument such as a multipart test or a multitest battery to assess the unique contribution of each component to the total test. The…

Factor loadings, used directly or as the basis of binary values, are not appropriate as weights to produce component scores from a rotated solution. A series of examples showing the results of an incorrect measurement of components is given. Several correlation matrices were taken from books on factor analysis and multivariate analysis. Each…

The effects of applying different methods of determining different sets of subscore weights on the composite score ranking of examinees were investigated. Four sets of subscore weights were applied to each of three examination results. The scores were from Advanced Placement (AP) Examinations in History of Art, Spanish Language, and Chemistry. One…

Several advantages to the use of factor scores as independent variables in a multiple regression equation were found. To help select the most desirable type of factor score on which to calculate a regression equation, computer-based Monte Carlo methods were used to compare the predictive accuracy upon replication of regression of five…

Computer-based Monte Carlo methods compared the predictive accuracy upon replication of regression of five complete and four incomplete factor score estimation methods. Prediction on incomplete factor scores showed better double cross-validated prediction accuracy than on complete scores. The unique unit-weighted factor score was superior among…

An investigation was conducted to determine the effects of response-category weighting and item weighting on reliability and predictive validity. Response-category weighting refers to scoring in which, for each category (including omit and "not read"), a weight is assigned that is proportional to the mean criterion score of examinees selecting…

PRIME is a library of 44 batch-oriented computer routines: 20 major package programs, which use 12 statistical utility routines, and 12 other utility routines for input/output and data manipulation. This manual contains a general description of data preparation and coding, standard control cards, input deck arrangement, standard options, and…

Previous research has studied the effects of different methods of item option weighting on the reliability and concurrent and predictive validity of achievement tests. Increases in reliability are generally found, but with mixed results for validity. Several methods of producing option weights, (i.e., Guttman internal and external weights and…

The purpose of this study was to determine whether option weighting improved the internal consistency and intercorrelation of the subtests. The differential option-weighting scheme employed in this study is based on one devised by Guttman. The tests were first scored with Guttman-type weights and then with conventional correction-for-guessing…

Technical improvements, which may be made in the reliability and validity of tests through confidence scores, are discussed. However, studies indicate that subjects do not handle their confidence uniformly. (MS)