We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found…

The present study explored the dimensionality of cognitive structure from two approaches. The first approach used a famous relation between Visual Spatial Working Memory (VSWM) and calculation to demonstrate the multidimensional item response analyses when true dimensions are unknown. The second approach explored the detectability of dimensions by…

How to compute and report subscores for a test that was originally designed for reporting scores on a unidimensional scale has been a topic of interest in recent years. In the research reported here, we describe an application of multidimensional item response theory to identify a subscore structure in a test designed for reporting results using a…

States are increasingly interested in including measures of student achievement growth, or "value- added," in evaluating teachers. Annual state assessments, however, which are the typical measure of student growth, usually cover only reading and math teachers and only in grades 4-8. These state assessments thus cannot …

States and school districts are exploring alternatives to state tests for measuring teachers' contributions to student learning. One approach applies statistical value-added methods to alternative student assessments such as commercially available tests and end-of course tests. The evidence suggests that these methods can reliably distinguish…

This paper challenges Hanushek and Woessmann's (2008) contention that the quality and not the quantity of schooling determines a nation's rate of economic growth. I first show that their statistical analysis is flawed. I then show that when a nation's average test scores and average schooling attainment are included in a national income model,…

Since the passage of the first Elementary and Secondary Education Act in 1965, increasing reading achievement for all students has been a focus for our nation. Unfortunately, our country still has over 30 million citizens who are illiterate (Mukherjee, 2007). One of the challenges for schools is to accurately identify students in need of early…

Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I…

This study sought to explore the relationship between attendance in public prekindergarten programs and school readiness skills using regression discontinuity methodology. A sample of 179 students entering prekindergarten and 67 students entering kindergarten who had completed prekindergarten the previous year was collected with parental consent…

The Internal Medicine In-Training Examination (ITE) is administered during residency training in the United States as a self-assessment and program assessment tool. Performance on this exam correlates with outcome on the American Board of Internal Medicine Certifying examination. Internal Medicine Program Directors use the United States Medical…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

This study aimed to provide evidence to guide decisions on the type and timing of assessments in a graduate medical programme, by identifying whether students from particular degree backgrounds face greater difficulty in satisfying the current assessment requirements. We examined the performance rank of students in three types of assessments and…

Mounting pressure in the policy arena to improve teacher productivity either by improving signals that predict teacher performance or through creating incentive contracts based on performance--has spurred two related questions: Are there important determinants of teacher productivity that are not captured by teacher credentials but that can be…

The accuracy with which regression models estimate treatment effects is dependent upon a number of conditions. The stability of the regression line (a function of sample size and correlation between pretest and posttest) is said to be the most important of these conditions. The utility of regression models is proportional to the size of the…

Reviews study that found, through use of a "performance instruction" model, a significant misalignment between the cognitive complexity levels of planned course content and that which is tested. (32 citations) (YKH)