Value-Added Models (VAMs) are both common and controversial in education policy and accountability research. While the sensitivity of VAMs to model specification and covariate selection is well documented, the extent to which test scoring methods (e.g., mean scores vs. IRT-based scores) may affect VA estimates is less studied. We examine the…

This paper examines one of three central diagnostic tools of the Concerns Based Adoption Model, the Stages of Concern Questionnaire (SoCQ). The SoCQ was developed with a focus on K12 education. It has been used widely since developed in 1973, in early childhood, higher education, medical, business, community, and military settings. The SoCQ…

The field of studies in metacognition points to some limitations in the way the construct has traditionally been measured and shows a near absence of performance-based tests. The Meta-Text is a performance-based test recently created to assess components of cognition regulation: planning, monitoring, and judgment. This study presents the first…

The importance of empowering school participants is increasing day by day. A school with empowered teachers can take more confident steps towards future education. Thus, it is important to determine the empowerment level of teachers as an important stakeholder of the school. The main purpose of this research is to adopt a measurement tool…

U.S. Supreme Court justices and other federal judges are, effectively, appointed for life, with no built-in check on their cognitive functioning as they approach old age. There is about a century of research on aging and intelligence that shows the vulnerability of processing speed, fluid reasoning, visual-spatial processing, and working memory to…

Abilities in scientific thinking and reasoning have been emphasized as core areas of initiatives, such as the Next Generation Science Standards or the College Board Standards for College Success in Science, which focus on the skills the future will demand of today's students. Although there is rich literature on studies of how these abilities…

Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

The Cognitive Style Indicator (CoSI) includes 3 cognitive dimensions: creating (flexible, open-ended and inventive), knowing (emphasizing facts, details, objectivity, and rationality), and planning (guided by preferences for certainty and well-structured information). The first aim of this research was to validate the 3-factor structure of the…

This quantitative pilot-study empirically tested the factor structure and reliability of the newly developed Graphical Inventory of Ethical Leadership (GIEL) scale, a Likert-style scale for quantifying key concepts related to ethical leadership. Correlations and principal components analyses were performed using pre-existing data as self-reports…

This study investigates the value of reporting the reading, listening, speaking, and writing section scores for the "TOEFL iBT"® test, focusing on 4 related aspects of the psychometric quality of the TOEFL iBT section scores: reliability of the section scores, dimensionality of the test, presence of distinct score profiles, and the…

Within the beta-binomial Bayesian framework, procedures are described for the evaluation of the kappa index of reliability on the basis of one administration of a domain-referenced test. Major factors affecting this index include cutoff score, test score variability and test length. Empirical data which substantiate some theoretical trends deduced…

A decision model designed to determine the reliability and validity of criterion referenced measures (CRMs) is presented. General procedures which pertain to the model are discussed as to: Measures of relationship, Reliability, Validity (content, criterion-oriented, and construct validation), and Item Analysis. The decision model is presented in…

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. (Editor)

Performance testing often entails the usage of expensive, time-consuming measures in the quest for determining the level of performance on some desired behavior. It is concluded that a generalizability theory approach to dealing with departures from reality in testing can aid in the establishment of empirically-based choices of measurement…

Symposium topics and speakers include "Choosing Needs for Needs Assessment" (Stephen P. Klein); "Selecting Tests to Assess the Needs" (Ralph Hoepfner); "Making Better Decisions on Assessed Needs: Differentiated School Norms" (Paul A. Bradley and Dale Woolley); and "Allocating Resources by Subject Area"…