Educational tests often have a cluster of items linked by a common stimulus ("testlet"). In such a design, the dependencies caused between items are called "testlet effects." In particular, the directional testlet effect (DTE) refers to a recursive influence whereby responses to earlier items can positively or negatively affect…

This paper examines one of three central diagnostic tools of the Concerns Based Adoption Model, the Stages of Concern Questionnaire (SoCQ). The SoCQ was developed with a focus on K12 education. It has been used widely since developed in 1973, in early childhood, higher education, medical, business, community, and military settings. The SoCQ…

This study aimed to address the need for a comprehensive assessment tool to evaluate the mathematical abilities of first-grade students through cognitive diagnostic assessment (CDA). The primary challenge involved in this endeavor was to delineate the specific cognitive skills and sub-skills pertinent to first-grade mathematics (FG-M) and to…

The development of assessments as part of the funded LOCUS project is described. The assessments measure students' conceptual understanding of statistics as outlined in the GAISE PreK-12 Framework. Results are reported from a large-scale administration to 3,430 students in grades 6 through 12 in the United States. Items were designed to assess…

Performance assessments, scenario-based tasks, and other groups of items carry a risk of violating the local item independence assumption made by unidimensional item response theory (IRT) models. Previous studies have identified negative impacts of ignoring such violations, most notably inflated reliability estimates. Still, the influence of this…

This article aims to give a comprehensive guide to planning and designing vocabulary tests which include Identifying the skills to be covered by the test; outlining the course content covered; preparing a table of specifications that shows the skill, content topics and number of questions allocated to each; and preparing the test instructions. The…

The main aims of this study were to further validate the Children's Social Understanding Scale (CSUS), which is a parent-report measure developed by Tahiroglu and colleagues, and to fill in some gaps in the existing research. Our study included more than 700 Polish parents from a diverse educational background who had children with disabilities,…

The extant literature has largely ignored a potentially significant source of variance in multiple mini-interview (MMI) scores by "hiding" the variance attributable to the sample of attributes used on an evaluation form. This potential source of hidden variance can be defined as rating items, which typically comprise an MMI evaluation…

Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

Undergraduate grade point average (GPA) is a commonly employed measure in educational research, serving as a criterion or as a predictor depending on the research question. Over the decades, researchers have used a variety of reliability coefficients to estimate the reliability of undergraduate GPA, which suggests that there has been no consensus…

Including growth models based on student test scores in teacher evaluations effectively holds teachers individually accountable for students improving their test scores. While an attractive policy for state administrators and advocates of education reform, value-added measures have been fraught with problems, and their use in teacher evaluation is…

Simulations are widely used. Simulations produce numbers that are deductive demonstrations of what a model says will happen.They produce numerical results that are consistent with the premises of the model used to generate the numbers. These simulated numerical results are not empirical data that address aspects of the world that lies outside the…

Houston's experience with the Educational Value-Added Assessment System (R) (EVAAS) raises questions that other districts should consider before buying the software and using it for high-stakes decisions. Researchers found that teachers in Houston, all of whom were under the EVAAS gun, but who taught relatively more racial minority students,…

The purpose of this study was to determine which of three competing models will provide, reliable, interpretable, and responsive measures of elementary students' understanding of the nature of science (NOS). The Nature of Science Instrument-Elementary (NOSI-E), a 28-item Rasch-based instrument, was used to assess students' NOS…

The Texas Assessment of Academic Skills (TAAS) test was the major source of data for the Texas educational accountability system from 1994 through 2002. Contrary to critics who claim that TAAS data are invalid and unreliable measures of student performance, structural equation analyses of TAAS reading data based on the 1994 Texas third grade…