The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a…

This is a keynote presentation given at AERA on developing a validity agenda for growth models in a large scale (e.g., state) setting. The emphasis of this presentation was to indicate that growth models and the validity agenda designed to provide evidence in supporting the claims to be made need to be personalized to meet the local or…

Looking beyond the right or wrong answer is imperative to the development of effective educational environments conducive to Pre-AP work in math. This presentation explores a system of evaluation in math that provides a personalized, student-reflective model correlated to consortia-based assessment. Using examples of students' work that includes…

Developing a cognitive model of task performance is an important and often overlooked phase in assessment design; failing to establish such a model can threaten the validity of the inferences made from the scores produced by an assessment (e.g., Leighton, 2004). Conducting think aloud interviews (TAIs), where students think aloud while completing…

On 9/13/12, the Workshop on Developing Assessments to Meet the Goals of the 2012 Framework for K-12 Science Education was held at the National Academies of Science. The workshop was organized and led by the NRC Committee on Developing Assessments of Science Proficiency in K-12 (co-chaired by James Pellegrino and Mark Wilson) and targeted to state…

There is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict first-year college performance at different institutions. This paper demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this variability. In a model that…

[Slides] presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes how the vehicles for gathering student evidence--task models and test specifications--are developed.

There is much debate on the merits and pitfalls of standardized tests for college admission, with questions regarding the format (multiple-choice versus constructed response), cognitive complexity, and content of these assessments (achievement versus aptitude) at the forefront of the discussion. This study addressed these questions by…

Presented at the Annual National Council on Measurement in Education (NCME) in New York in March 2008. This presentation explores the growth trajectory of the SAT Reading scores and examine what demographics and variation may cause changes and affect growth.

Presented at the national conference for AERA (American Educational Research Association) in April 2009. The large variability of SAT taker population across states makes state-by-state comparisons of the SAT scores challenging. Using a mixture modeling approach, therefore, the current study presents a method of identifying subpopulations in terms…

[Slides] presented at the annual conference of the National Council of Supervisors of Math.

The "Educational Media and Technology Yearbook" is dedicated to theoretical, empirical and practical approaches to educational media development. All chapters are invited and selected based on a variety of strategies to determine current trends and issues in the field. The 2009 edition will highlight innovative Trends and Issues in…