In this article, the authors applied profile analysis to Maths exam data to demonstrate how different exam forms, differing in difficulty and length, can be reported and easily interpreted. The results were presented for different groups of participants and for different institutions in different Maths domains by evaluating the balance. Some…

Evaluation is an essential component of any Extension education program. One tool, the pre- and post-test, provides measurable evaluation data. Yet often the answer "I don't know" or all possible answers to a multiple choice question are not included in the repeated measure analysis. Because more than two answers are offered, the test of marginal…

Test developers are responsible to define how test scores should be interpreted and used. The No Child Left Behind Act of 2001 (NCLB) directed the Secretary of Education to use results from the National Assessment of Educational Progress (NAEP) to confirm the proficiency scores from state developed tests. There are two sets of federal definitions…

The National Council for Accreditation of Teacher Education (NCATE, 2002) requires teacher education units to develop assessment systems and evaluate both the success of candidates and unit operations. Because of a stated, but misguided, fear of statistics, NCATE fails to use accepted terminology to assure the quality of institutional evaluative…

In keeping with the theme of the International Language Testing Association/Language Testing Research Colloquium Conference in 2008, "Focusing on the Core: Justifying the Use of Language Assessments to Stakeholders," I define "stakeholder-friendly tests," "defensible testing," and "testing-context analysis."…

This module prepares the reader to use statistical procedures to detect differentially functioning test items. The Mantel-Haenszel statistic, logistic regression, the SIBTEST procedure, the Standardization procedure, and various item response theory-based procedures are presented. Theoretical frameworks, strengths and weaknesses, and…

Educational test theory consists of statistical and methodological tools to support inferences about examinees' knowledge, skills, and accomplishments. The evolution of test theory has been shaped by the nature of users' inferences which, until recently, have been framed almost exclusively in terms of trait and behavioral psychology. Progress in…

The present paper suggests that multivariate techniques are very important in social science research, and that canonical correlation analysis may be particularly useful. The logic of canonical analysis is explained and discussed. The necessity of using replicability/generalizability analyses is argued. It is suggested that cross-validation…

Five of the common misuses of tests are reviewed: (1) acceptance of the test title as an accurate and complete description of the variable being measured (failure to examine the manual and the items carefully to know the specific aspects to be tested can result in misuse through selection of an inappropriate test for a particular purpose or…

Sternberg and Gastel have provided an example of a major principle of intelligence research--the relationship between a task's working memory demands and its sensitivity to individual differences in fluid intelligence and "g." There is no need to invoke additional constructs such as "novelty." (SLD)

It is asserted that the statistical arguments proposed by Humphreys are not supported by the formula he presents. The disagreement with Larson is with the contention that there exists one correct theoretical framework in which intelligence research should be posed. The respective viewpoints are seen as complementary, not competing. (SLD)

Principal components analysis of standardization sample data for the Comprehensive Test of Adaptive Behavior, involving 6,647 persons with mental retardation, found that a single factor accounted for 86 percent of the variance in category scores. Use of the Total Score and idiographic item clusters is psychometrically preferable to use of category…

For anchor test equating, 3 linear observed score methods are derived for populations differing in ability. Each version requires that the correlations of the tests with the selection variable be known. Five sets of assumptions are made for each model--yielding 15 methods--which are then related to existing methods. (SLD)

When the Educational Testing Service became the administrator of the National Assessment of Educational Progress (NAEP) in 1983, it introduced scales based on item response theory (IRT) as a way of presenting results of the assessment to the general public. Some properties of the scales and their uses are discussed. Initial attempts at presenting…

An explicit model identifying 10 relevant components of achievement gain scores has been developed. Based on that model, all students under consideration are stratified according to individual observed pretest score, and achievement gains are measured relative to the average and range of gains among students in the same prescore stratum. The…