This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

The present study investigated the concurrent validity of the Classroom Strategies Scale-Teacher Form (CSS-T), a multidimensional teacher formative assessment of instructional and behavioral management practices. The CSS-T is compared with the Classroom Assessment Scoring System (CLASS), a well-known teacher assessment of overall classroom…

This report introduces the "WorkFORCE"™ Assessment for Job Fit, a personality assessment utilizing the "FACETS"™ core capability, which is based on innovations in forced-choice assessment and computer adaptive testing. The instrument is derived from the fivefactor model (FFM) of personality and encompasses a broad spectrum of…

In early 2001, to support an internal evaluation of the impact of changing exclusion rates on reports of statistically significant gains across states, the National Center for Education Statistics (NCES) sponsored research on imputation procedures of National Assessment of Educational Progress (NAEP) scores for the excluded students and provided…

In recent years, the Education for All movement has focused more intensely on the quality of education, rather than simply provision. Many recent and current education quality interventions focus on literacy, which is the core skill required for further academic success. Despite this focus on the quality of literacy instruction in developing…

Three descriptive statistics (difficulty, discrimination, and reliability) of multiple-choice (MC) test items were compared to those of a new (1980s) format of machine-scored questions. The new method, answer-bank multi-digit testing (MDT), uses alphabetized lists of up to 1,000 alternatives and approximates the completion style of assessment…

The strategy for investigating convergent and discriminant test validity, known as the multitrait-multimethod matrix, is investigated. A nonparametric significance testing procedure is suggested and demonstrated. (JKS)

The partitioned-U procedure is outlined, a fundamental logical flaw in this procedure's avoidance of any direct test of the significance of the first discriminant function or largest coefficient of canonical correlation is pointed out, and two alternatives to the partitioned-U procedure are discussed. (Author/DEP)

The known interval scale, referred to as the 7.8 scale, has been criticized as an invalid measuring instrument in the form of an attitude scale. It is the purpose of this paper to demonstrate that this scale can produce spuriously inflated correlation coefficients, high reliability, and false significance on statistical tests. The case will be…

The problem of determining the significance level which should be used in statistical tests of item validity in order to minimize type I errors is discussed. (Author/JKS)

Attempts to distill a growing technical literature on repeated-measures analysis of variance into a few simple principles for selecting an analytic technique. Argues that researchers ought not opt for a general analysis strategy when current computer technology makes it possible to select the optimal analysis technique for a given data set. (RH)

A test of statistical significance is a procedure for determining how likely a result is assuming a null hypothesis to be true with randomization and a sample of size n (the given size in the study). Randomization, which refers to random sampling and random assignment, is important because it ensures the independence of observations, but it does…

A multitrait-multimethod matrix containing two methods of measuring 12 personality traits was analyzed and confirmatory factor analysis was applied to the data. Although unexplained variance remained, method factors and a general personality factor significantly improved the fit of a model containing only trait factors. (Author/RL)

The intent of this investigation was to perform a validation study to determine whether RACE (Racial Attitude and Cultural Expression test) differentiates between primary grade students identified as having negative and positive attitudes. Students were categorized by a combination of administrator, teacher and clinical assessment into a negative…

The predictive validity of scores on the National Board of Medical Examiners (NBME) Part I and Part II examinations for the selection of residents in orthopaedic surgery was investigated. Use of NBME scores has been criticized because of the time lag between taking Part I and entering residency and because Part I content is not directly linked to…